Ms-swift

2. Web-UI 支持音频和视频。
3. 支持使用 OpenAI API 部署音频和视频模型。
4. 采用新的多模态训练框架。
5. 支持视频模型的推理加速（lmdeploy 和 internvl2 系列）。

新模型：
1. idefics3-8b-llama3
2. llava-hf 72b、110b、llama3-llava
3. deepseek-coder-v2、deepseek-coder-lite-v2、deepseek-v2

What's Changed
* support video deploy by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1632
* fix infer deploy by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1665
* Fix bugs by tastelikefeet in https://github.com/modelscope/ms-swift/pull/1664
* Add generation config support for swift eval by Yunnglin in https://github.com/modelscope/ms-swift/pull/1659
* Add a bunch of small features by tastelikefeet in https://github.com/modelscope/ms-swift/pull/1669
* Support lmdeploy&vllm support video by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1668
* [TorchAcc] fix: add mark_step when GA is greater than 1 by baoleai in https://github.com/modelscope/ms-swift/pull/1677
* Support audio and video in web-ui by tastelikefeet in https://github.com/modelscope/ms-swift/pull/1681
* Fix lmdeploy deploy seed by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1678
* add paper link by tastelikefeet in https://github.com/modelscope/ms-swift/pull/1684
* set default writer_batch_size argument in mllm rlhf by hjh0119 in https://github.com/modelscope/ms-swift/pull/1689
* New multimodal framework by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1594
* fix video component by tastelikefeet in https://github.com/modelscope/ms-swift/pull/1692
* refactor internvl2 by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1625
* fix model dtype in rlhf by hjh0119 in https://github.com/modelscope/ms-swift/pull/1696
* Fix minicpmv 2.6 & video/plain-text bug by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1698
* Fix internvl2 device map by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1701
* Support deepseek-v2 base by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1708
* [TorchAcc] update data bucketing strategy by baoleai in https://github.com/modelscope/ms-swift/pull/1706
* [TorchAcc] consolidate FSDP ckpt after saving instead of during loading by baoleai in https://github.com/modelscope/ms-swift/pull/1709
* fix deploy stream by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1713
* update docs by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1714
* Fix mllm rlhf by hjh0119 in https://github.com/modelscope/ms-swift/pull/1707
* support idefics3 by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1720
* Fix minicpm-v-v2.6 device_map='auto' & agent deploy by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1733
* Fix kto by hjh0119 in https://github.com/modelscope/ms-swift/pull/1737
* support llava-hf 72b, 110b, llama3-llava-hf by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/1741
* Fix glm4v rlhf by hjh0119 in https://github.com/modelscope/ms-swift/pull/1745

New Contributors
* Yunnglin made their first contribution in https://github.com/modelscope/ms-swift/pull/1659

**Full Changelog**: https://github.com/modelscope/ms-swift/compare/v2.3.0...v2.3.1

中文版

新特性
1. 支持多模态大模型SequenceClassification架构用于多模态分类任务，参考[这里](https://github.com/modelscope/ms-swift/blob/main/examples/train/seq_cls/qwen2_vl/sft.sh)。
2. 支持多模态大模型reward model训练。

新模型
1. Shanghai_AI_Laboratory/internlm3-8b-instruct
2. OpenBMB/MiniCPM-o-2_6
3. deepseek-ai/DeepSeek-R1, deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B系列
4. bytedance-research/Valley-Eagle-7B
5. LLM-Research/phi-4
6. Qwen/Qwen2.5-Math-PRM-7B, Qwen/Qwen2.5-Math-PRM-72B
7. MiniMaxAI/MiniMax-Text-01, MiniMaxAI/MiniMax-VL-01

English Version

New Features

1. Support multi-modal large model SequenceClassification architecture for multi-modal classification tasks, see [here](https://github.com/modelscope/ms-swift/blob/main/examples/train/seq_cls/qwen2_vl/sft.sh).
2. Support training of multi-modal reward model.

New Models

1. Shanghai_AI_Laboratory/internlm3-8b-instruct
2. OpenBMB/MiniCPM-o-2_6
3. deepseek-ai/DeepSeek-R1, deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B series
4. bytedance-research/Valley-Eagle-7B
5. LLM-Research/phi-4
6. Qwen/Qwen2.5-Math-PRM-7B, Qwen/Qwen2.5-Math-PRM-72B
7. MiniMaxAI/MiniMax-Text-01, MiniMaxAI/MiniMax-VL-01




What's Changed
* update qlora shell by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2880
* fix docs by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2882
* support multi round dpo by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2884
* Support infer n parameter by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2893
* Fix qwen vl eval by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2892
* fix infer engine by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2898
* Add phi4 by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2895
* fix link & bug by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2902
* update video infer examples by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2840
* Sampler by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2905
* Fix a bug when lint code by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2906
* Fix bugs by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2907
* update plugin doc by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2908
* fix vllm tp stuck by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2909
* fix replace_video2image by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2913
* Fix read file mode by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2915
* fix inspect init by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2916
* Update rm by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2919
* Add internlm3 dense by HIT-cwh in https://github.com/modelscope/ms-swift/pull/2920
* internlm3 lint pass by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2923
* Fix web ui log by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2924
* Support Valley by lxline in https://github.com/modelscope/ms-swift/pull/2921
* support minicpm-o by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2918
* fix vllm tp block by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2927
* update docs by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2929
* Support first prms by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2926
* fix Valley by lxline in https://github.com/modelscope/ms-swift/pull/2931
* Support mllm seq_cls/rm by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2934
* fix bugs by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2938
* support deepseek-ai/DeepSeek-R1 by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2940
* Fix quant template by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2942
* Support minimax by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2943
* Fix mllm seq cls by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2945
* support deepseek_r1_distill by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2946
* fix demo_hf by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2951
* fix infer_stream by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2952
* fix citest by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2953
* fix bugs by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2954
* update requirements by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2957
* update web-ui images by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2958
* update quant_mllm shell by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2959
* fix max_length error print by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2960
* fix seq_cls patcher by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2963
* ppo compat transformers>=4.47.* by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2964


**Full Changelog**: https://github.com/modelscope/ms-swift/compare/v3.0.2...v3.0.3

中文版

新特性
1. 支持使用`swift app`开启可视化推理创空间，参考[这里](https://github.com/modelscope/ms-swift/tree/main/examples/app)
2. 支持大模型的RM和PPO训练，参考[这里](https://github.com/modelscope/ms-swift/tree/main/examples/train/rlhf)
3. 支持`SequenceClassification`模型（含BERT）的BNB/GPTQ量化，参考[这里](https://github.com/modelscope/ms-swift/tree/main/examples/export/quantize/bert)
4. 支持reward model的推理、部署和BNB/GPTQ量化

新模型
1. ZhipuAI/cogagent-9b-20241220
2. Reward Models: Shanghai_AI_Laboratory/internlm2-1_8b-reward系列, Qwen/Qwen2-Math-RM-72B系列, AI-ModelScope/Skywork-Reward-Llama-3.1-8B系列, AI-ModelScope/GRM_Llama3.1_8B_rewardmodel-ft系列
3. AIDC-AI/Ovis1.6-Gemma2-27B, AIDC-AI/Ovis1.6-Llama3.2-3B
4. PowerInfer/SmallThinker-3B-Preview

新数据集
1. PowerInfer/LONGCOT-Refine-500K, PowerInfer/QWQ-LONGCOT-500K

English Version

New Features

1. Support for using `swift app` to launch a visual inference creative space, see [here](https://github.com/modelscope/ms-swift/tree/main/examples/app)
2. Support for RM and PPO training of large models, see [here](https://github.com/modelscope/ms-swift/tree/main/examples/train/rlhf)
3. Support for BNB/GPTQ quantization of `SequenceClassification` models (including BERT), see [here](https://github.com/modelscope/ms-swift/tree/main/examples/export/quantize/bert)
4. Support for inference, deployment, and BNB/GPTQ quantization of reward models

New Models

1. ZhipuAI/cogagent-9b-20241220
2. Reward Models: Shanghai_AI_Laboratory/internlm2-1_8b-reward series, Qwen/Qwen2-Math-RM-72B series, AI-ModelScope/Skywork-Reward-Llama-3.1-8B series, AI-ModelScope/GRM_Llama3.1_8B_rewardmodel-ft series
3. AIDC-AI/Ovis1.6-Gemma2-27B, AIDC-AI/Ovis1.6-Llama3.2-3B
4. PowerInfer/SmallThinker-3B-Preview

New Datasets

1. PowerInfer/LONGCOT-Refine-500K, PowerInfer/QWQ-LONGCOT-500K

What's Changed
* Fix app-ui dropdown by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2787
* fix multi-lora by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2790
* fix stream infer by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2793
* fix some web-ui bugs by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2794
* support swift app by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2792
* fix pt batch infer by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2800
* fix world_size by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2801
* update base_model deploy example by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2803
* fix glm4v by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2806
* fix swift deploy log error (repeat log) by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2808
* support ZhipuAI/cogagent-9b-20241220 by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2810
* fix citest by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2812
* fix enable_cache by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2813
* update docs (specific model arguments) by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2822
* add 'right' option for 'truncation_strategy' by zsxm1998 in https://github.com/modelscope/ms-swift/pull/2754
* Fix glm4v suffix by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2829
* Update padding side by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2832
* Update base_to_chat shell by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2833
* Fix bugs by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2838
* Fix some bugs by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2848
* support reward_model by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2849
* Move optimizer to create_optimizer by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2851
* fix post_init by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2855
* fix cache_name_file by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2856
* fix telechat template by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2857
* Update more models by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2852
* Support quant bert reward by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2859
* fix jsonl writer by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2860
* support reward model train by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2862
* fix vllm video by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2864
* support mps by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2866
* Update agent demo by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2867
* fix bugs by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2869
* Support ppo by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2783
* update citest by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2873
* fix dataset cache bugs by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2876

New Contributors
* zsxm1998 made their first contribution in https://github.com/modelscope/ms-swift/pull/2754

**Full Changelog**: https://github.com/modelscope/ms-swift/compare/v3.0.1...v3.0.2

中文版

新特性
1. 支持`SequenceClassification`模型的训练、推理和部署。可以查看以下例子：[qwen2.5](https://github.com/modelscope/ms-swift/tree/main/examples/train/seq_cls/qwen2_5)，[bert](https://github.com/modelscope/ms-swift/tree/main/examples/train/seq_cls/bert)。
2. LlamaPro支持多模态模型. 例如：qwen2vl、internvl2.5、llama3-vision等。

新模型
1. Qwen/QVQ-72B-Preview
2. iic/DocOwl2
3. OpenGVLab/InternVL2-Pretrain-Models, OpenGVLab/InternVL2_5-4B-AWQ系列, OpenGVLab/InternVL2_5-1B-MPO系列
4. deepseek-ai/DeepSeek-V3系列
5. answerdotai/ModernBERT-base系列
6. AI-ModelScope/paligemma2-3b-pt-224系列, AI-ModelScope/paligemma2-3b-ft-docci-448系列
7. AI-ModelScope/Skywork-o1-Open-Llama-3.1-8B

English Version

New Features:
1. Support for training, inference, and deployment of `SequenceClassification` models. You can check the following examples: [qwen2.5](https://github.com/modelscope/ms-swift/tree/main/examples/train/seq_cls/qwen2_5), [bert](https://github.com/modelscope/ms-swift/tree/main/examples/train/seq_cls/bert).
2. LlamaPro supports multimodal models, such as qwen2vl, internvl2.5, and llama3-vision.

New Models:
1. Qwen/QVQ-72B-Preview
2. iic/DocOwl2
3. OpenGVLab/InternVL2-Pretrain-Models, OpenGVLab/InternVL2_5-4B-AWQ series, OpenGVLab/InternVL2_5-1B-MPO series
4. deepseek-ai/DeepSeek-V3 series
5. answerdotai/ModernBERT-base series
6. AI-ModelScope/paligemma2-3b-pt-224 series, AI-ModelScope/paligemma2-3b-ft-docci-448 series
7. AI-ModelScope/Skywork-o1-Open-Llama-3.1-8B

What's Changed
* Fix mplug owl2, molmo by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2724
* fix batch_infer pad_token & florence by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2725
* Support qwen agent format by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2722
* Support more internvl2.5 awq/mpo & internvl2 pretrain model by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2726
* support iic/DocOwl2 by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2728
* update examples by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2730
* remove files by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2732
* support paligemma2 by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2735
* fix windows by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2733
* support multi-modal llamapro by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2738
* support AI-ModelScope/Skywork-o1-Open-Llama-3.1-8B by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2739
* Fix windows encoding gbk by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2741
* fix docs multimodal by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2742
* support SequenceClassification & update QVQ-72B-Preview by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2747
* fix web-ui by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2758
* fix bugs by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2761
* fix shell by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2764
* fix app-ui by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2765
* support modern_bert & support bert deploy by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2767
* fix alpaca by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2771
* support txt by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2772
* fix telechat2 template by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2775
* Fix deepspeed by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2778
* fix qwen2vl by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2779
* Fix app ui by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2780
* support deepseek-v3 by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2781
* Fix app-ui by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2784


**Full Changelog**: https://github.com/modelscope/ms-swift/compare/v3.0.0...v3.0.1

中文版

架构修改与新特性：
具体可以查看这里: https://swift.readthedocs.io/zh-cn/latest/Instruction/ReleaseNote3.0.html

新模型：
1. OpenGVLab/InternVL2_5-1B等系列模型
2. LLM-Research/Llama-3.3-70B-Instruct
3. BAAI/Emu3-Gen
4. deepseek-ai/DeepSeek-V2.5-1210, deepseek-ai/deepseek-vl2等系列模型
5. Shanghai_AI_Laboratory/internlm-xcomposer2d5-ol-7b
6. InfiniAI/Megrez-3b-Instruct, InfiniAI/Megrez-3B-Omni
7. TeleAI/TeleChat2-3B等系列模型

English Version

Architecture Modifications and New Features:
For more details, please visit: https://swift.readthedocs.io/en/latest/Instruction/ReleaseNote3.0.html

New Models:
1. OpenGVLab/InternVL2_5-1B series models
2. LLM-Research/Llama-3.3-70B-Instruct
3. BAAI/Emu3-Gen
4. deepseek-ai/DeepSeek-V2.5-1210, deepseek-ai/deepseek-vl2 series models
5. Shanghai_AI_Laboratory/internlm-xcomposer2d5-ol-7b
6. InfiniAI/Megrez-3b-Instruct, InfiniAI/Megrez-3B-Omni
7. TeleAI/TeleChat2-3B series models

What's Changed
* Refactor All Codes and bump version to 3.0 by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2030
* fix doc by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2545
* fix manifest by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2546
* add doc 2.x by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2548
* fix ui by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2549
* fix infer by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2550
* Refactor mllm by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2543
* fix ui by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2552
* Fix ui by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2556
* Update ddp infer doc by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2557
* fix docs by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2558
* Fix docs by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2561
* fix log by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2564
* Fix the command line parameter doc by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2565
* fix context by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2568
* Documents Updates by yrk111222 in https://github.com/modelscope/ms-swift/pull/2574
* Revert "Documents Updates" by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2576
* fix hub param by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2572
* Fix bugs by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2573
* Support internvl2.5 by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2575
* update english docs by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2577
* fix en docs by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2580
* fix docs & add custom example by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2581
* fix custom example by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2582
* support llama3.3 by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2584
* update acc_strategy & fix citest by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2583
* Support peft0.14 by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2587
* update infer/deploy examples by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2588
* add image images mapping by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2594
* update llm sft notebook by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2599
* fix notebook by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2600
* Fix streaming by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2601
* Emu3 gen train by mi804 in https://github.com/modelscope/ms-swift/pull/2602
* compat mllm notebook by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2604
* Temporarily remove torchacc. by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2606
* update docs by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2607
* train and infer scripts for emu3_gen by mi804 in https://github.com/modelscope/ms-swift/pull/2610
* Uodate Document by yrk111222 in https://github.com/modelscope/ms-swift/pull/2615
* update memory usage of emu3-gen by mi804 in https://github.com/modelscope/ms-swift/pull/2611
* move prepare_model by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2614
* Update mllm notebook by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2617
* Support all-embedding / all-norm by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2619
* fix lmdeploy==0.5.* by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2621
* Support deepseek-ai/DeepSeek-V2.5-1210 by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2624
* fix use_reentrant gradient_checkpointing by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2625
* support reward model by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2628
* fix add_default_tag by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2631
* fix dataset by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2636
* fix bugs & update openbuddy models & update docs by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2638
* fix app-ui by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2641
* Fix post encode by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2643
* fix bugs by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2645
* update truncation_strategy by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2647
* fix swift/Infinity-Instruct by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2651
* Support LoRA-GA by lxline in https://github.com/modelscope/ms-swift/pull/2650
* support deepseek_vl2 by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2654
* fix swift/SlimOrca by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2656
* fix swift/SlimOrca by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2657
* support Shanghai_AI_Laboratory/internlm-xcomposer2d5-ol-7b:audio by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2658
* support Shanghai_AI_Laboratory/internlm-xcomposer2d5-ol-7b:base by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2660
* fix hub by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2661
* fix liger by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2666
* support megrez by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2667
* fix unsloth resume training by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2668
* fix dataset by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2670
* Fix bugs by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2671
* fix deepseek_vl2 by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2675
* support adapters by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2633
* Support megrez omni by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2674
* fix docs by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2679
* fix megrez_omni by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2680
* fix infer by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2681
* Fix bugs by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2687
* Update readme by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2579
* update wechat by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2694
* fix readme by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2696
* Fix web-ui by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2693
* Fix readme by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2697
* Update banner by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2699
* fix use_reentrant by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2700
* update examples by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2703
* fix eval strategy by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2707
* Update FAQ by slin000111 in https://github.com/modelscope/ms-swift/pull/2706
* qwen to Qwen by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2708
* fix timeout & web-ui by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2709
* Fix multi lora by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2711
* support Qwen/QVQ-72B-Preview by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2712
* update examples by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2714
* fix deploy request_config by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2718
* fix examples by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2719
* fix gptq group_size by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2720
* Better error messages by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2721

New Contributors
* yrk111222 made their first contribution in https://github.com/modelscope/ms-swift/pull/2574
* lxline made their first contribution in https://github.com/modelscope/ms-swift/pull/2650

**Full Changelog**: https://github.com/modelscope/ms-swift/compare/v2.6.1...v3.0.0

New Models:
1. [Marco-o1](https://www.modelscope.cn/models/AIDC-AI/Marco-o1)
2. [mPLUG-Owl3-7B-241101](https://www.modelscope.cn/models/iic/mPLUG-Owl3-7B-241101)
3. [QwQ-32B-Preview](https://www.modelscope.cn/models/Qwen/QwQ-32B-Preview)
4. [glm-edge](ZhipuAI/glm-edge-4b-chat), [glm-edge-v](ZhipuAI/glm-edge-v-5b)

New Datasets:
1. [OpenO1-SFT](https://www.modelscope.cn/datasets/AI-ModelScope/OpenO1-SFT)

What's Changed
* support part tuner replace_key False by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2438
* bump ms version by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2449
* remove useless code by tastelikefeet in https://github.com/modelscope/ms-swift/pull/2453
* fix qwen2-vl position_ids by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2461
* fix peft is_multimodal by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2462
* fix qwen2vl pt infer by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2463
* [TorchAcc] Update padding strategy when using persistent cache by eedalong in https://github.com/modelscope/ms-swift/pull/2464
* fix kto by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2478
* Update Common QA by slin000111 in https://github.com/modelscope/ms-swift/pull/2475
* fix awq quant device_map by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2488
* Fix preprocess num proc by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2492
* Support marco o1 by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2496
* fix eval_dataset no by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2497
* support batch flattening collator by eedalong in https://github.com/modelscope/ms-swift/pull/2499
* fix latex-ocr by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2510
* support mPLUG-Owl3 241101 by LukeForeverYoung in https://github.com/modelscope/ms-swift/pull/2515
* support qwq by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2520
* support glm-edge & glm-edge-v by Jintao-Huang in https://github.com/modelscope/ms-swift/pull/2526

New Contributors
* eedalong made their first contribution in https://github.com/modelscope/ms-swift/pull/2464

**Full Changelog**: https://github.com/modelscope/ms-swift/compare/v2.6.0...v2.6.1