Llamafactory

Stronger [LlamaBoard](https://github.com/hiyouga/LLaMA-Factory?tab=readme-ov-file#fine-tuning-with-llama-board-gui-powered-by-gradio) 💪😀

- Support single-node distributed training in Web UI
- Add dropdown menu for easily resuming from checkpoints and picking saved configurations by hiyouga and hzhaoy in 4053
- Support selecting checkpoints of full/freeze tuning
- Add throughput metrics to LlamaBoard by injet-zhou in 4066
- Faster UI loading

New features

- Add KTO algorithm by enji-zhou in 3785
- Add SimPO algorithm by hiyouga
- Support passing `max_lora_rank` to the vLLM backend by jue-jue-zi in 3794
- Support preference datasets in sharegpt format and remove big files from git repo by hiyouga in 3799
- Support setting system messages in CLI inference by ycjcl868 in 3812
- Add `num_samples` option in `dataset_info.json` by seanzhang-zhichen in 3829
- Add NPU docker image by dongdongqiang2018 in 3876
- Improve NPU document by MengqingCao in 3930
- Support SFT packing with greedy knapsack algorithm by AlongWY in 4009
- Add `llamafactory-cli env` for bug report
- Support image input in the API mode
- Support random initialization via the `train_from_scratch` argument
- Initialize CI

New models

- Base models
- Qwen2 (0.5B/1.5B/7B/72B/MoE) 📄
- PaliGemma-3B (pt/mix) 📄🖼️
- GLM-4-9B 📄
- Falcon-11B 📄
- DeepSeek-V2-Lite (16B) 📄
- Instruct/Chat models
- Qwen2-Instruct (0.5B/1.5B/7B/72B/MoE) 📄🤖
- Mistral-7B-Instruct-v0.3 📄🤖
- Phi-3-small-8k-instruct (7B) 📄🤖
- Aya-23 (8B/35B) 📄🤖
- OpenChat-3.6-8B 📄🤖
- GLM-4-9B-Chat 📄🤖
- TeleChat-12B-Chat by hzhaoy in 3958 📄🤖
- Phi-3-medium-8k-instruct (14B) 📄🤖
- DeepSeek-V2-Lite-Chat (16B) 📄🤖
- Codestral-22B-v0.1 📄🤖

New datasets

- Pre-training datasets
- FineWeb (en)
- FineWeb-Edu (en)
- Supervised fine-tuning datasets
- Ruozhiba-GPT4 (zh)
- STEM-Instruction (zh)
- Preference datasets
- Argilla-KTO-mix-15K (en)
- UltraFeedback (en)

Bug fix

- Fix RLHF for multimodal finetuning
- Fix LoRA target in multimodal finetuning by BUAADreamer in 3835
- Fix `yi` template by Yimi81 in 3925
- Fix abort issue in LlamaBoard by injet-zhou in 3987
- Pass `scheduler_specific_kwargs` to `get_scheduler` by Uminosachi in 4006
- Fix hyperparameters helps by xu-song in 4007
- Update issue template by statelesshz in 4011
- Fix vllm dtype parameter
- Fix exporting hyperparameters by MengqingCao in 4080
- Fix DeepSpeed ZeRO3 in PPO trainer
- Fix 3108 3387 3646 3717 3764 3769 3803 3807 3818 3837 3847 3853 3873 3900 3931 3965 3971 3978 3992 4005 4012 4013 4022 4033 4043 4061 4075 4077 4079 4085 4090 4120 4132 4137 4139

🚨🚨 Core refactor 🚨🚨

- Add **CLIs** usage, now we recommend using `llamafactory-cli` to launch training and inference, the entry point is located at the [cli.py](https://github.com/hiyouga/LLaMA-Factory/blob/main/src/llamafactory/cli.py)
- Rename files: `train_bash.py` -> `train.py`, `train_web.py` -> `webui.py`, `api_demo.py` -> `api.py`
- Remove files: `cli_demo.py`, `evaluate.py`, `export_model.py`, `web_demo.py`, use `llamafactory-cli chat/eval/export/webchat` instead
- Use **YAML configs** in examples instead of shell scripts for a pretty view
- Remove the sha1 hash check when loading datasets
- Rename arguments: `num_layer_trainable` -> `freeze_trainable_layers`, `name_module_trainable` -> `freeze_trainable_modules`

The above changes are made by hiyouga in 3596

REMINDER: Now [installation](https://github.com/hiyouga/LLaMA-Factory?tab=readme-ov-file#installation) is **mandatory** to use LLaMA Factory

New features

- Support training and inference on the Ascend NPU 910 devices by zhou-wjjw and statelesshz (docker images are also provided)
- Support `stop` parameter in vLLM engine by zhaonx in 3527
- Support fine-tuning token embeddings in freeze tuning via the `freeze_extra_modules` argument
- Add Llama3 [quickstart](https://github.com/hiyouga/LLaMA-Factory?tab=readme-ov-file#quickstart) to readme

New models

- Base models
- Yi-1.5 (6B/9B/34B) 📄
- DeepSeek-V2 (236B) 📄
- Instruct/Chat models
- Yi-1.5-Chat (6B/9B/34B) 📄🤖
- Yi-VL-Chat (6B/34B) by BUAADreamer in 3748 📄🖼️🤖
- Llama3-Chinese-Chat (8B/70B) 📄🤖
- DeepSeek-V2-Chat (236B) 📄🤖

Bug fix

- Add badam arguments to LlamaBoard by codemayq in 3487
- Add openai data format to readme by khazic in 3490
- Fix slow operation in dpo/orpo trainer by hiyouga
- Fix badam examples by pha123661 in 3578
- Fix download link of the nectar_rm dataset by ZeyuTeng96 in 3588
- Add project by Katehuuh in 3601
- Fix dockerfile by gaussian8 in 3604
- Fix full tuning of MLLMs by BUAADreamer in 3651
- Fix gradio environment variables by cocktailpeanut in 3654
- Fix typo and add log in API by Tendo33 in 3655
- Fix download link of the phi-3 model by YUUUCC in 3683
- Fix 3559 3560 3602 3603 3606 3625 3650 3658 3674 3694 3702 3724 3728

Congratulations on 20k stars 🎉 We are the 1st of the *GitHub Trending* at Apr. 23rd 🔥 Follow us at *[X](https://twitter.com/llamafactory_ai)*

New features

- Support SFT/PPO/DPO/ORPO for the **LLaVA-1.5** model by BUAADreamer in 3450
- Support inferring the LLaVA-1.5 model with both native Transformers and vLLM by hiyouga in 3454
- Support **vLLM+LoRA** inference for partial models (see [support list](https://docs.vllm.ai/en/latest/models/supported_models.html))
- Support 2x faster generation of the QLoRA model based on [UnslothAI](https://github.com/unslothai/unsloth)'s optimization
- Support adding new special tokens to the tokenizer via the `new_special_tokens` argument
- Support choosing the device to merge LoRA in LlamaBoard via the `export_device` argument
- Add a Colab notebook for getting into fine-tuning the Llama-3 model on a free T4 GPU
- Automatically enable SDPA attention and fast tokenizer for higher performance

New models

- Base models
- OLMo-1.7-7B
- Jamba-v0.1-51B
- Qwen1.5-110B
- DBRX-132B-Base
- Instruct/Chat models
- Phi-3-mini-3.8B-instruct (4k/128k)
- LLaVA-1.5-7B
- LLaVA-1.5-13B
- Qwen1.5-110B-Chat
- DBRX-132B-Instruct

New datasets

- Supervised fine-tuning datasets
- LLaVA mixed (en&zh) by BUAADreamer in 3471
- Preference datasets
- DPO mixed (en&zh) by hiyouga

Bug fix

- Fix 2093 3333 3347 3374 3387

New features

- Support Meta Llama-3 (8B/70B) models
- Support [UnslothAI](https://github.com/unslothai/unsloth)'s long-context QLoRA optimization (56,000 context length for Llama-2 7B in 24GB)
- Support previewing local datasets in directories in LlamaBoard by codemayq in 3291

New algorithms

- Support [BAdam](https://arxiv.org/abs/2404.02827) algorithm by Ledzy in #3287
- Support [Mixture-of-Depths](https://arxiv.org/abs/2404.02258) training by mlinmg in #3338

New models

- Base models
- CodeGemma (2B/7B)
- CodeQwen1.5-7B
- Llama-3 (8B/70B)
- Mixtral-8x22B-v0.1
- Instruct/Chat models
- CodeGemma-7B-it
- CodeQwen1.5-7B-Chat
- Llama-3-Instruct (8B/70B)
- Command R (35B) by marko1616 in 3254
- Command R+ (104B) by marko1616 in 3254
- Mixtral-8x22B-Instruct-v0.1

Bug fix

- Fix full-tuning batch prediction examples by khazic in 3261
- Fix output_router_logits of Mixtral by liu-zichen in 3276
- Fix automodel from pretrained with attn implementation (see https://github.com/huggingface/transformers/issues/30298)
- Fix unable to convergence issue in the layerwise galore optimizer (see https://github.com/huggingface/transformers/issues/30371)
- Fix 3184 3238 3247 3273 3316 3317 3324 3348 3352 3365 3366

New features

- Support **[ORPO](https://arxiv.org/abs/2403.07691)** algorithm by hiyouga in #3066
- Support inferring BNB 4-bit models on multiple GPUs via the `quantization_device_map` argument
- Reorganize README files, move example scripts to the `examples` folder
- Support saving & loading arguments quickly in LlamaBoard by hiyouga and marko1616 in 3046
- Support load alpaca-format dataset from the hub without `dataset_info.json` by specifying `--dataset_dir ONLINE`
- Add a parameter `moe_aux_loss_coef` to control the coefficient of auxiliary loss in MoE models.

New models

- Base models
- Breeze-7B-Base
- Qwen1.5-MoE-A2.7B (14B)
- Qwen1.5-32B
- Instruct/Chat models
- Breeze-7B-Instruct
- Qwen1.5-MoE-A2.7B-Chat (14B)
- Qwen1.5-32B-Chat

Bug fix

- Fix pile dataset download config by lealaxy in 3053
- Fix model generation config by marko1616 in 3057
- Fix qwen1.5 models DPO training by changingivan and hiyouga in 3083
- Support Qwen1.5-32B by sliderSun in 3160
- Support Breeze-7B by codemayq in 3161
- Fix `addtional_target` in unsloth by kno10 in 3201
- Fix 2807 3022 3023 3046 3077 3085 3116 3200 3225

This patch mainly fixes 2983

In commit 9bec3c98a22c91b1c28fda757db51eb780291641, we built the optimizer and scheduler inside the trainers, which inadvertently introduced a bug: when DeepSpeed was enabled, the trainers in transformers would build an optimizer and scheduler before calling the `create_optimizer_and_scheduler` method [1], then the optimizer created by our method would overwrite the original one, while the scheduler would not. Consequently, the scheduler would no longer affect the learning rate in the optimizer, leading to a regression in the training result. We have fixed this bug in 3bcd41b639899e72bcabc51d59bac8967af19899 and 8c77b1091296e204dc3c8c1f157c288ca5b236bd. Thank HideLord for helping us identify this critical bug.

[1] https://github.com/huggingface/transformers/blob/v4.39.1/src/transformers/trainer.py#L1877-L1881

We have also fixed 2961 2981 2982 2983 2991 3010