Ms-swift

New Features:
1. Added support for swift export, enabling awq-int4 quantization and gpt-int2,3,4,8 quantization. Models can be pushed to the Modelscope Hub. You can view the documentation [here](https://github.com/modelscope/swift/blob/main/docs/source/LLM/LLM%E9%87%8F%E5%8C%96%E6%96%87%E6%A1%A3.md).
2. Enabled fine-tuning of [awq quantized models](https://github.com/modelscope/swift/blob/main/examples/pytorch/llm/scripts/qwen1half_7b_chat_awq/lora/sft.sh).
3. Enabled fine-tuning of [aqlm quantized models](https://github.com/modelscope/swift/blob/main/examples/pytorch/llm/scripts/llama2_7b_aqlm_2bit_1x16/lora/sft.sh).
4. Added support for deploying LLM with infer_backend='pt'.
5. Added web-ui with task management and visualization of training loss, eval loss, etc. Inference is accelerated using VLLM.

New Tuners:
1. Lora+.
2. LlamaPro.

New Models:
1. qwen1.5 awq series.
2. gemma series.
3. yi-9b.
4. deepseek-math series.
5. internlm2-1_8b series.
6. openbuddy-mixtral-moe-7b-chat.
7. llama2 aqlm series.

New Datasets:
1. ms-bench-mini.
2. hh-rlhf-cn series.
3. disc-law-sft-zh, disc-med-sft-zh.
4. pileval.


What's Changed
* Fix vllm==0.3 deploy bug by Jintao-Huang in https://github.com/modelscope/swift/pull/412
* Support deepseek math by Jintao-Huang in https://github.com/modelscope/swift/pull/413
* update support_vllm by Jintao-Huang in https://github.com/modelscope/swift/pull/415
* fix zero3 & swift lora by Jintao-Huang in https://github.com/modelscope/swift/pull/416
* Support peft0.8.0 by tastelikefeet in https://github.com/modelscope/swift/pull/423
* update readme by Jintao-Huang in https://github.com/modelscope/swift/pull/426
* fix pai open with 'a' by Jintao-Huang in https://github.com/modelscope/swift/pull/430
* default load_best_model_at_end=False by Jintao-Huang in https://github.com/modelscope/swift/pull/432
* support openbuddy mixtral by Jintao-Huang in https://github.com/modelscope/swift/pull/437
* support gemma by Jintao-Huang in https://github.com/modelscope/swift/pull/441
* Support ms bench mini by Jintao-Huang in https://github.com/modelscope/swift/pull/442
* Add roadmap and contributing doc by tastelikefeet in https://github.com/modelscope/swift/pull/431
* support peft format by tastelikefeet in https://github.com/modelscope/swift/pull/438
* update contributing.md by Jintao-Huang in https://github.com/modelscope/swift/pull/446
* fix link by tastelikefeet in https://github.com/modelscope/swift/pull/447
* Fix rlhf dataset by tastelikefeet in https://github.com/modelscope/swift/pull/451
* Add task management for webui by tastelikefeet in https://github.com/modelscope/swift/pull/457
* Support swift export by Jintao-Huang in https://github.com/modelscope/swift/pull/455
* Fix llm quantization docs by Jintao-Huang in https://github.com/modelscope/swift/pull/458
* fix get_vllm_engine bug by Jintao-Huang in https://github.com/modelscope/swift/pull/463
* use cpu export by Jintao-Huang in https://github.com/modelscope/swift/pull/462
* Fix llama2 generation config by Jintao-Huang in https://github.com/modelscope/swift/pull/468
* Support editing model_id_or_path by tastelikefeet in https://github.com/modelscope/swift/pull/469
* Support pt deploy by Jintao-Huang in https://github.com/modelscope/swift/pull/467
* Fix swift deploy bug by Jintao-Huang in https://github.com/modelscope/swift/pull/470
* fix deploy dep by Jintao-Huang in https://github.com/modelscope/swift/pull/471
* Support LLaMAPRO and LoRA+ by tastelikefeet in https://github.com/modelscope/swift/pull/472
* Support internlm2 1.8b by Jintao-Huang in https://github.com/modelscope/swift/pull/473
* fix deepseek moe device_map by Jintao-Huang in https://github.com/modelscope/swift/pull/476
* fix peft compatible bug by tastelikefeet in https://github.com/modelscope/swift/pull/482
* Fix deepspeed init bug by Jintao-Huang in https://github.com/modelscope/swift/pull/481
* fix scripts docs by Jintao-Huang in https://github.com/modelscope/swift/pull/483
* Update `swift export` and update docs by Jintao-Huang in https://github.com/modelscope/swift/pull/484
* support gptq export by Jintao-Huang in https://github.com/modelscope/swift/pull/485
* fix docs & readme by Jintao-Huang in https://github.com/modelscope/swift/pull/486
* fix app-ui bug by Jintao-Huang in https://github.com/modelscope/swift/pull/488
* Support peft0.9 by tastelikefeet in https://github.com/modelscope/swift/pull/490
* support torchrun_args for dpo cli and support web_ui model deployment by slin000111 in https://github.com/modelscope/swift/pull/496
* Support transformers 4.33.0 by tastelikefeet in https://github.com/modelscope/swift/pull/498
* Update deepspeed config by Jintao-Huang in https://github.com/modelscope/swift/pull/500
* move docs to classroom by tastelikefeet in https://github.com/modelscope/swift/pull/503
* Support yi 9b by Jintao-Huang in https://github.com/modelscope/swift/pull/504
* Update yi sh by Jintao-Huang in https://github.com/modelscope/swift/pull/506


**Full Changelog**: https://github.com/modelscope/swift/compare/v1.6.0...v1.7.0

New Models:
1. deepseek-math series

New Datasets:
1. sharegpt-gpt4-mini
2. disc-law-sft-zh
3. disc-med-sft-zh

Bug Fix
1. Fix vllm==0.3 & swift deploy bug.
2. Fix zero3 & swift lora bug.

**Full Changelog**: https://github.com/modelscope/swift/compare/v1.6.0...v1.6.1

New Features:
1. Agent Training
2. AIGC support: controlnet, controlnet_sdxl, dreambooth, text_to_image, text_to_image_sdxl
3. Compatibility with vllm==0.3.*

New Models:
1. qwen1.5 series
2. openbmb series


What's Changed
* update openbmb sh by Jintao-Huang in https://github.com/modelscope/swift/pull/361
* Fix openbmb model name by tastelikefeet in https://github.com/modelscope/swift/pull/362
* support dpo cli and add examples controlnet and dreambooth by slin000111 in https://github.com/modelscope/swift/pull/344
* support openbmb minicpm by Jintao-Huang in https://github.com/modelscope/swift/pull/364
* Support agent training, etc. by tastelikefeet in https://github.com/modelscope/swift/pull/352
* fix tuner by tastelikefeet in https://github.com/modelscope/swift/pull/365
* Fix agent doc by tastelikefeet in https://github.com/modelscope/swift/pull/366
* Fix data format in readme by tastelikefeet in https://github.com/modelscope/swift/pull/367
* fix lazy_tokenize bug by Jintao-Huang in https://github.com/modelscope/swift/pull/369
* Fix length penalty by Jintao-Huang in https://github.com/modelscope/swift/pull/371
* fix loss by tastelikefeet in https://github.com/modelscope/swift/pull/372
* update compute loss by Jintao-Huang in https://github.com/modelscope/swift/pull/375
* fix system='' bug by Jintao-Huang in https://github.com/modelscope/swift/pull/374
* fix system='' bug by Jintao-Huang in https://github.com/modelscope/swift/pull/378
* Support PAI compat by Jintao-Huang in https://github.com/modelscope/swift/pull/373
* fix doc by tastelikefeet in https://github.com/modelscope/swift/pull/376
* Fix the conflict between agent and CT by tastelikefeet in https://github.com/modelscope/swift/pull/379
* fix cogagent_18b_chat sh typo error by Jintao-Huang in https://github.com/modelscope/swift/pull/381
* Fix loss scale by tastelikefeet in https://github.com/modelscope/swift/pull/383
* Feat/qwen1.5 by tastelikefeet in https://github.com/modelscope/swift/pull/385
* fix template name by tastelikefeet in https://github.com/modelscope/swift/pull/389
* update readme by Jintao-Huang in https://github.com/modelscope/swift/pull/386
* update readme by Jintao-Huang in https://github.com/modelscope/swift/pull/390
* Support max model len by Jintao-Huang in https://github.com/modelscope/swift/pull/392
* Support vllm max model len by Jintao-Huang in https://github.com/modelscope/swift/pull/394
* fix arguments bug by Jintao-Huang in https://github.com/modelscope/swift/pull/395
* support vllm 0.3 by Jintao-Huang in https://github.com/modelscope/swift/pull/396
* fix deepspeed_config_path bug by Jintao-Huang in https://github.com/modelscope/swift/pull/398
* fix file name by slin000111 in https://github.com/modelscope/swift/pull/397
* Add qwen1.5 scripts by tastelikefeet in https://github.com/modelscope/swift/pull/393
* fix many bugs by Jintao-Huang in https://github.com/modelscope/swift/pull/399
* Fix baichuan2 int4 bug by Jintao-Huang in https://github.com/modelscope/swift/pull/400
* Fix qwen1half deploy bug by Jintao-Huang in https://github.com/modelscope/swift/pull/402
* fix readme and test_llm by tastelikefeet in https://github.com/modelscope/swift/pull/404
* update readme by Jintao-Huang in https://github.com/modelscope/swift/pull/405


**Full Changelog**: https://github.com/modelscope/swift/compare/v1.5.4...v1.6.0

New Features:
1. Default zero3.json file
2. Enhanced support for multi-modal models

New Models:
1. Orion series
2. Codefuse series
3. Internlm2-math series
4. Internlm2 series
5. Qwen2 series
6. Yi-vl series
7. Internlm-xcomposer2

What's Changed
* Update orion 14b by Jintao-Huang in https://github.com/modelscope/swift/pull/341
* update codefuse series by Jintao-Huang in https://github.com/modelscope/swift/pull/343
* Support yi vl by Jintao-Huang in https://github.com/modelscope/swift/pull/345
* fix yi-vl finetune error by Yimi81 in https://github.com/modelscope/swift/pull/347
* Fix template encode bug by Jintao-Huang in https://github.com/modelscope/swift/pull/348
* Update internlm2 math by Jintao-Huang in https://github.com/modelscope/swift/pull/349
* Removing eos_token when doing inference. by Jintao-Huang in https://github.com/modelscope/swift/pull/351
* Support zero3 by Jintao-Huang in https://github.com/modelscope/swift/pull/353
* Support internlm xcomposer2 by Jintao-Huang in https://github.com/modelscope/swift/pull/354
* update qwen2 by Jintao-Huang in https://github.com/modelscope/swift/pull/355
* fix baichuan2 bug by Jintao-Huang in https://github.com/modelscope/swift/pull/357
* fix template_encode bug by Jintao-Huang in https://github.com/modelscope/swift/pull/358
* Fix issue 342 by Jintao-Huang in https://github.com/modelscope/swift/pull/359
* fix test_run.py by Jintao-Huang in https://github.com/modelscope/swift/pull/360

New Contributors
* Yimi81 made their first contribution in https://github.com/modelscope/swift/pull/347

**Full Changelog**: https://github.com/modelscope/swift/compare/v1.5.3...v1.5.4

English Version

1. Support show log in text box of web-ui
2. Support share=True in web-ui, only need to set WEBUI_SHARE=1 in environment variable
3. Support deactivate all adapters
4. Support more SFT arguments
5. Add longlora/qalora script
6. Support custom models in web-ui
7. ModelScope SWIFT studio released: https://www.modelscope.cn/studios/damo/Scalable-lightWeight-Infrastructure-for-Fine-Tuning/summary
8. Fix some bugs

----------
中文版本
1. 支持在web-ui中直接显示日志
2. 支持share=True 仅需要在环境变量中设置WEBUI_SHARE=1
3. 支持失活所有adapters
4. 添加了更多SFT参数
5. 添加了longlora/qalora的训练脚本
6. web-ui支持了自己注册的自定义模型
7. SWFT魔搭创空间上线了: https://www.modelscope.cn/studios/damo/Scalable-lightWeight-Infrastructure-for-Fine-Tuning/summary
8. 修复了一些bug

What's Changed
* fix chatglm3 template bug by Jintao-Huang in https://github.com/modelscope/swift/pull/298
* Support studio by tastelikefeet in https://github.com/modelscope/swift/pull/300
* fix text label by tastelikefeet in https://github.com/modelscope/swift/pull/301
* fix a bug may cause module on gpu throws error by tastelikefeet in https://github.com/modelscope/swift/pull/302
* fix_ziya_template_bug by Jintao-Huang in https://github.com/modelscope/swift/pull/303


**Full Changelog**: https://github.com/modelscope/swift/compare/v1.5.1...v1.5.2

English version

New Features

1. Support dtype settings in LoRA
2. Support deactivated tuners offloading
3. Support deployment with OpenAI format restful API
4. Make LongLoRA supports the latest llama2 code

新feature
1. 支持LoRA设置dtype类型
2. 支持将不使用的tuners offloading到cpu和meta设备上
3. 支持OpenAI restful API方式的部署
4. LongLoRA支持最新的llama2代码

What's Changed
* update docs by tastelikefeet in https://github.com/modelscope/swift/pull/269
* Update benchmark 0101 by Jintao-Huang in https://github.com/modelscope/swift/pull/271
* update benchmark by Jintao-Huang in https://github.com/modelscope/swift/pull/276
* Support dtype in lora by tastelikefeet in https://github.com/modelscope/swift/pull/278
* Support deploy by Jintao-Huang in https://github.com/modelscope/swift/pull/275
* update deploy client by Jintao-Huang in https://github.com/modelscope/swift/pull/279
* Support offload by tastelikefeet in https://github.com/modelscope/swift/pull/281
* fix tuner bug by Jintao-Huang in https://github.com/modelscope/swift/pull/285
* Support lora_modules_to_save by Jintao-Huang in https://github.com/modelscope/swift/pull/284
* update template by Jintao-Huang in https://github.com/modelscope/swift/pull/286
* support ModuleToSave original module offloading by tastelikefeet in https://github.com/modelscope/swift/pull/282
* Fix offload by tastelikefeet in https://github.com/modelscope/swift/pull/288
* fix scedit bug by tastelikefeet in https://github.com/modelscope/swift/pull/290
* fix bnb qwen bug by Jintao-Huang in https://github.com/modelscope/swift/pull/289
* update readme by Jintao-Huang in https://github.com/modelscope/swift/pull/291
* update readme by Jintao-Huang in https://github.com/modelscope/swift/pull/292
* Fix/longlora by tastelikefeet in https://github.com/modelscope/swift/pull/294
* support additional_trainable_parameters by Jintao-Huang in https://github.com/modelscope/swift/pull/295
* fix webui by tastelikefeet in https://github.com/modelscope/swift/pull/296
* fix trainer push to hub by Jintao-Huang in https://github.com/modelscope/swift/pull/297


**Full Changelog**: https://github.com/modelscope/swift/compare/v1.5.0...v1.5.1