Llamafactory

Congratulations on 20k stars 🎉 We are the 1st of the *GitHub Trending* at Apr. 23rd 🔥 Follow us at *[X](https://twitter.com/llamafactory_ai)*

New features

- Support SFT/PPO/DPO/ORPO for the **LLaVA-1.5** model by BUAADreamer in 3450
- Support inferring the LLaVA-1.5 model with both native Transformers and vLLM by hiyouga in 3454
- Support **vLLM+LoRA** inference for partial models (see [support list](https://docs.vllm.ai/en/latest/models/supported_models.html))
- Support 2x faster generation of the QLoRA model based on [UnslothAI](https://github.com/unslothai/unsloth)'s optimization
- Support adding new special tokens to the tokenizer via the `new_special_tokens` argument
- Support choosing the device to merge LoRA in LlamaBoard via the `export_device` argument
- Add a Colab notebook for getting into fine-tuning the Llama-3 model on a free T4 GPU
- Automatically enable SDPA attention and fast tokenizer for higher performance

New models

- Base models
- OLMo-1.7-7B
- Jamba-v0.1-51B
- Qwen1.5-110B
- DBRX-132B-Base
- Instruct/Chat models
- Phi-3-mini-3.8B-instruct (4k/128k)
- LLaVA-1.5-7B
- LLaVA-1.5-13B
- Qwen1.5-110B-Chat
- DBRX-132B-Instruct

New datasets

- Supervised fine-tuning datasets
- LLaVA mixed (en&zh) by BUAADreamer in 3471
- Preference datasets
- DPO mixed (en&zh) by hiyouga

Bug fix

- Fix 2093 3333 3347 3374 3387

New features

- Support Meta Llama-3 (8B/70B) models
- Support [UnslothAI](https://github.com/unslothai/unsloth)'s long-context QLoRA optimization (56,000 context length for Llama-2 7B in 24GB)
- Support previewing local datasets in directories in LlamaBoard by codemayq in 3291

New algorithms

- Support [BAdam](https://arxiv.org/abs/2404.02827) algorithm by Ledzy in #3287
- Support [Mixture-of-Depths](https://arxiv.org/abs/2404.02258) training by mlinmg in #3338

New models

- Base models
- CodeGemma (2B/7B)
- CodeQwen1.5-7B
- Llama-3 (8B/70B)
- Mixtral-8x22B-v0.1
- Instruct/Chat models
- CodeGemma-7B-it
- CodeQwen1.5-7B-Chat
- Llama-3-Instruct (8B/70B)
- Command R (35B) by marko1616 in 3254
- Command R+ (104B) by marko1616 in 3254
- Mixtral-8x22B-Instruct-v0.1

Bug fix

- Fix full-tuning batch prediction examples by khazic in 3261
- Fix output_router_logits of Mixtral by liu-zichen in 3276
- Fix automodel from pretrained with attn implementation (see https://github.com/huggingface/transformers/issues/30298)
- Fix unable to convergence issue in the layerwise galore optimizer (see https://github.com/huggingface/transformers/issues/30371)
- Fix 3184 3238 3247 3273 3316 3317 3324 3348 3352 3365 3366

New features

- Support **[ORPO](https://arxiv.org/abs/2403.07691)** algorithm by hiyouga in #3066
- Support inferring BNB 4-bit models on multiple GPUs via the `quantization_device_map` argument
- Reorganize README files, move example scripts to the `examples` folder
- Support saving & loading arguments quickly in LlamaBoard by hiyouga and marko1616 in 3046
- Support load alpaca-format dataset from the hub without `dataset_info.json` by specifying `--dataset_dir ONLINE`
- Add a parameter `moe_aux_loss_coef` to control the coefficient of auxiliary loss in MoE models.

New models

- Base models
- Breeze-7B-Base
- Qwen1.5-MoE-A2.7B (14B)
- Qwen1.5-32B
- Instruct/Chat models
- Breeze-7B-Instruct
- Qwen1.5-MoE-A2.7B-Chat (14B)
- Qwen1.5-32B-Chat

Bug fix

- Fix pile dataset download config by lealaxy in 3053
- Fix model generation config by marko1616 in 3057
- Fix qwen1.5 models DPO training by changingivan and hiyouga in 3083
- Support Qwen1.5-32B by sliderSun in 3160
- Support Breeze-7B by codemayq in 3161
- Fix `addtional_target` in unsloth by kno10 in 3201
- Fix 2807 3022 3023 3046 3077 3085 3116 3200 3225

This patch mainly fixes 2983

In commit 9bec3c98a22c91b1c28fda757db51eb780291641, we built the optimizer and scheduler inside the trainers, which inadvertently introduced a bug: when DeepSpeed was enabled, the trainers in transformers would build an optimizer and scheduler before calling the `create_optimizer_and_scheduler` method [1], then the optimizer created by our method would overwrite the original one, while the scheduler would not. Consequently, the scheduler would no longer affect the learning rate in the optimizer, leading to a regression in the training result. We have fixed this bug in 3bcd41b639899e72bcabc51d59bac8967af19899 and 8c77b1091296e204dc3c8c1f157c288ca5b236bd. Thank HideLord for helping us identify this critical bug.

[1] https://github.com/huggingface/transformers/blob/v4.39.1/src/transformers/trainer.py#L1877-L1881

We have also fixed 2961 2981 2982 2983 2991 3010

We released our [paper](https://arxiv.org/abs/2403.13372) on arXiv! Thanks to all co-authors and [AK's recommendation](https://twitter.com/_akhaliq/status/1770660136391946656)

New features

- Support **GaLore** algorithm, allowing full-parameter learning of a 7B model using less than 24GB VRAM
- Support **FSDP+QLoRA** that allows QLoRA fine-tuning of a 70B model on 2x24GB GPUs
- Support **LoRA+** algorithm for better LoRA fine-tuning by qibaoyuan in 2830
- LLaMA Factory 🤝 [vLLM](https://github.com/vllm-project/vllm), enjoy **270%** inference speed with `--infer_backend vllm`
- Add Colab [notebook](https://colab.research.google.com/drive/1eRTPn37ltBbYsISy9Aw2NuI2Aq5CQrD9?usp=sharing) for easily getting started
- Support pushing fine-tuned models to Hugging Face Hub in web UI
- Support `apply_chat_template` by adding a chat template to the tokenizer after fine-tuning
- Add dockerize support by S3Studio in 2743 2849

New models

- Base models
- OLMo (1B/7B)
- StarCoder2 (3B/7B/15B)
- Yi-9B
- Instruct/Chat models
- OLMo-7B-Instruct

New datasets

- Supervised fine-tuning datasets
- Cosmopedia (en)
- Preference datasets
- Orca DPO (en)

Bug fix

- Fix flash_attn in web UI by cx2333-gt in 2730
- Fix deepspeed runtime error in PPO by stephen-nju in 2746
- Fix readme ddp instruction by khazic in 2903
- Fix environment variable in datasets by SirlyDreamer in 2905
- Fix readme information by 0xez in 2919
- Fix generation config validation by marko1616 in 2945
- Fix requirements by rkinas in 2963
- Fix bitsandbytes windows version by Tsumugii24 in 2967
- Fix 2346 2642 2649 2732 2735 2756 2766 2775 2777 2782 2798 2802 2803 2817 2895 2928 2936 2941

New features

- Support **[DoRA](https://arxiv.org/abs/2402.09353)** (Weight-Decomposed LoRA)
- Support QLoRA for the AWQ/AQLM quantized models, now **2-bit QLoRA** is feasible
- Provide some example scripts in https://github.com/hiyouga/LLaMA-Factory/tree/main/examples

New models

- Base models
- Gemma (2B/7B)
- Instruct/Chat models
- Gemma-it (2B/7B)

Bug fix

- Add flash-attn package for Windows user by codemayq in 2514
- Fix ppo trainer 1163 by stephen-nju in 2525
- Support atom models by Rayrtfr in 2531
- Support role in webui by lungothrin in 2575
- Bump accelerate to 0.27.2 and fix 2552 by Katehuuh in 2608
- Fix 2512 2516 2532 2533 2629