Llamafactory

New features

- Support full-parameter RLHF training (RM & PPO)
- Refactor llmtuner core in 1525 by hiyouga
- Better LLaMA Board: full-parameter RLHF and demo mode

New models

- Base models
- ChineseLLaMA-1.3B
- LingoWhale-8B
- Instruct/Chat models
- ChineseAlpaca-1.3B
- Zephyr-7B-Alpha/Beta

Bug fix

- Fix bugs in partial-parameter (freeze) tuning
- Fix 224 336 931 936 1011 1489 1494 1507 1514

Bug fix

- Fix the OOM issue in PPO training by mmbwf in 424
- Fix fine-tuning arguments by yyq in 1454
- Refactor constants and evaluation by hiyouga
- Fix 1452 1466 1478

New features

- Support [**NEFTune**](https://arxiv.org/abs/2310.05914) trick for supervised fine-tuning by anvie in #1252
- Support loading dataset in the sharegpt format - read [data/readme](https://github.com/hiyouga/LLaMA-Factory/blob/main/data/README.md) for details
- Support generating multiple responses in demo API via the `n` parameter
- Support caching the pre-processed dataset files via the `cache_path` argument
- Better LLaMA Board (pagination, controls, etc.)
- Support `push_to_hub` argument 1088

New models

- Base models
- ChatGLM3-6B-Base
- Yi (6B/34B)
- Mistral-7B
- BlueLM-7B-Base
- Skywork-13B-Base
- XVERSE-65B
- Falcon-180B
- Deepseek-Coder-Base (1.3B/6.7B/33B)
- Instruct/Chat models
- ChatGLM3-6B
- Mistral-7B-Instruct
- BlueLM-7B-Chat
- Zephyr-7B
- OpenChat-3.5
- Yayi (7B/13B)
- Deepseek-Coder-Instruct (1.3B/6.7B/33B)

New datasets

- Pre-training datasets
- RedPajama V2
- Pile
- Supervised fine-tuning datasets
- OpenPlatypus
- ShareGPT Hyperfiltered
- ShareGPT4
- UltraChat 200k
- AgentInstruct
- LMSYS Chat 1M
- Evol Instruct V2

Bug fix

- Fix full-parameter DPO training 1383 1422 (inspired by mengban )
- Fix tokenizer config by lvzii in 1436
- Fix 1197 1215 1217 1218 1228 1232 1285 1287 1290 1316 1325 1349 1356 1365 1411 1418 1438 1439 1446

New features

- Support [**LongLoRA**](https://github.com/dvlab-research/LongLoRA) for the LLaMA models
- Support training the Qwen-14B and InternLM-20B models
- Support training state recovery for the all-in-one Web UI
- Support Ascend NPU by statelesshz in 975
- Integrate MMLU, C-Eval and CMMLU benchmarks

Modifications

- Rename repository to **LLaMA Factory** (former LLaMA Efficient Tuning)
- Use the `cutoff_len` argument instead of `max_source_length` and `max_target_length` 944
- Add a `train_on_prompt` option 1184

Bug fix

- Fix numeric error caused by the layer norm dtype in https://github.com/hiyouga/LLaMA-Factory/commit/84b7486885c600e5e65c5ba9095d56ecc2502977 [1]
- Fix bugs in PPO Trainer by mmbwf in 900
- Fix 424 762 814 887 913 1000 1026 1032 1064 1068 1074 1086 1097 1176 1177 1190 1191

[1] https://github.com/huggingface/transformers/pull/25598#discussion_r1335345914

New features

- Support [**FlashAttention-2**](https://github.com/Dao-AILab/flash-attention) for LLaMA models. (RTX4090, A100, A800 or H100 GPU is required)
- Support training the Baichuan2 models
- Use right-padding to avoid overflow in fp16 training (also mentioned [here](https://github.com/huggingface/trl/blob/22a90198e5e2652161dbf955e196a97033decf40/examples/research_projects/stack_llama_2/scripts/sft_llama2.py#L168))
- Align the computation method of the reward score with DeepSpeed-Chat (better generation)
- Support `--lora_target all` argument which automatically finds the applicable modules for LoRA training

Bug fix

- Use efficient EOS tokens to align with the Baichuan training ( https://github.com/baichuan-inc/Baichuan2/issues/23 )
- Remove PeftTrainer to save model checkpoints in DeepSpeed training
- Fix bugs in web UI by beat4ocean in 596 by codemayq in 644 651 678 741 by kinghuin in 786
- Add dataset explanation by panpan0000 in 629
- Fix a bug in the DPO data collator
- Fix a bug of the ChatGLM2 tokenizer in right-padding
- 608 617 649 757 761 763 809 818

New features

- Preview training script in Web UI by codemayq in 479 511
- Support resuming from checkpoints by niuba in 434 (`transformers>=4.31.0` required)
- Two RoPE scaling methods: linear and NTK-aware scaling for LLaMA models (`transformers>=4.31.0` required)
- Support training the ChatGLM2-6B model
- Support PPO training in bfloat16 data type 551

Bug fix

- Unusual output of quantized models 278 391
- Runtime error in distributed DPO training 480
- Unexpected truncation in generation 532
- Dataset streaming error in pre-training 548 549
- Tensor shape mismatch in PPO training using ChatGLM2 527 528
- 475 476 478 481 494 551