Alpaca-eval

What's Changed
* [ENH] add strict decoding OAI by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/394
* Add blendaxai-gm-l6-vo31 to AlpacaEval by ym-blendax-ai in https://github.com/tatsu-lab/alpaca_eval/pull/399
* Added Llama3-PBM-Nova-70B model by PKU-Baichuan in https://github.com/tatsu-lab/alpaca_eval/pull/395
* Add evaluator weighted_alpaca_eval_gpt-4o-mini-2024-07-18 by tongyx361 in https://github.com/tatsu-lab/alpaca_eval/pull/401
* Add Shopee-SlimMoA-v1 to AlpacaEval by LLM-Alignment-sh in https://github.com/tatsu-lab/alpaca_eval/pull/398
* [ENH] add metadata to completion: date, version,... by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/402
* Add REBEL-Llama-3-8B-Instruct-Armo to AlpacaEval by ZhaolinGao in https://github.com/tatsu-lab/alpaca_eval/pull/403
* Add Llama-3-8B-Instruct-SkillMix to AlpacaEval by parksimon0808 in https://github.com/tatsu-lab/alpaca_eval/pull/405
* Updated HF Link in model_configs for Llama-3-8B-Instruct-SkillMix by parksimon0808 in https://github.com/tatsu-lab/alpaca_eval/pull/409
* Add SelfMoA_gemma-2-9b-it-SimPO, SelfMoA_gemma-2-9b-it-WPO-HB to AlpacaEval by wenzhe-li in https://github.com/tatsu-lab/alpaca_eval/pull/411
* add Self-taught-llama3.1-70B-dpo as a evaluator by tianlu-wang in https://github.com/tatsu-lab/alpaca_eval/pull/412
* Add GPO-Llama-3-8B-Instruct-GPM-2B and SPPO-Llama-3-8B-Instruct-GPM-2… by xukp20 in https://github.com/tatsu-lab/alpaca_eval/pull/413
* Add NullModel to AlpacaEval by xszheng2020 in https://github.com/tatsu-lab/alpaca_eval/pull/414
* Add Llama-3-Instruct-8B-RainbowPO to AlpacaEval by hanyang1999 in https://github.com/tatsu-lab/alpaca_eval/pull/416
* add example for Llama3 vllm server by cameron-chen in https://github.com/tatsu-lab/alpaca_eval/pull/404
* Add FuseChat-3.0 models to AlpacaEval by yangzy39 in https://github.com/tatsu-lab/alpaca_eval/pull/426
* Add TOA to AlpacaEval by oceanypt in https://github.com/tatsu-lab/alpaca_eval/pull/428
* [BUG] tool_calls by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/429

New Contributors
* PKU-Baichuan made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/395
* LLM-Alignment-sh made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/398
* parksimon0808 made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/405
* wenzhe-li made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/411
* tianlu-wang made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/412
* xukp20 made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/413
* xszheng2020 made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/414
* hanyang1999 made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/416
* cameron-chen made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/404
* yangzy39 made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/426
* oceanypt made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/428

**Full Changelog**: https://github.com/tatsu-lab/alpaca_eval/compare/v0.6.5...v0.6.6

What's Changed
* Add Llama-3-Instruct-8B-WPO-HB-v2 to AlpacaEval by wzhouad in https://github.com/tatsu-lab/alpaca_eval/pull/377
* [ENH] add llama 3.1 by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/378
* [ENH] add example for LLama 3 vllm by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/381
* Add Infinity-Instruct-7M-0729-Llama3_1-70B, Infinity-Instruct-7M-0729-Llama3_1-8B, Infinity-Instruct-7M-0729-mistral-7B to AlpacaEval by cszhengyh in https://github.com/tatsu-lab/alpaca_eval/pull/383
* Add gemma-2-9b-it-WPO-HB to AlpacaEval by wzhouad in https://github.com/tatsu-lab/alpaca_eval/pull/384
* Add link to gemma-2-9b-it-WPO-HB by wzhouad in https://github.com/tatsu-lab/alpaca_eval/pull/385
* Change the name of the Infinity-Instruct-7M-0729-Models to Infinity-Instruct-7M-Gen-Models by cszhengyh in https://github.com/tatsu-lab/alpaca_eval/pull/387
* Add blendaxai-gm-l3-v35 to AlpacaEval by ym-blendax-ai in https://github.com/tatsu-lab/alpaca_eval/pull/389
* [ENH] OpenAI use tools instead of functions by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/391
* [ENH] enable base_dir to be a list by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/392
* [ENH] add mistral v0.3, Qwen2 70b, gtp4 mini by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/393

New Contributors
* wzhouad made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/377
* ym-blendax-ai made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/389

**Full Changelog**: https://github.com/tatsu-lab/alpaca_eval/compare/v0.6.4...v0.6.5

What's Changed
* Add SPPO-Llama-3-Instruct-8B-PairRM to AlpacaEval by Edward-Sun in https://github.com/tatsu-lab/alpaca_eval/pull/354
* Add Infinity-Instruct-3M-0613-Llama3-70B to AlpacaEval by cszhengyh in https://github.com/tatsu-lab/alpaca_eval/pull/358
* Add SPPO-Gemma-2-9B-It-PairRM to AlpacaEval by angelahzyuan in https://github.com/tatsu-lab/alpaca_eval/pull/359
* Add Infinity-Instruct-3M-0625-Models to AlpacaEval by cszhengyh in https://github.com/tatsu-lab/alpaca_eval/pull/364
* Add Higgs Llama3-70B V2 Results by sxjscience in https://github.com/tatsu-lab/alpaca_eval/pull/367
* Added Ghost 8B Beta (d0x5) model by lh0x00 in https://github.com/tatsu-lab/alpaca_eval/pull/366
* Add gemma-2-9b-it-SimPO and gemma-2-9b-it-DPO to AlpacaEval by xiamengzhou in https://github.com/tatsu-lab/alpaca_eval/pull/368
* [ENH] add CI test for unwanted files by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/369
* update model links by xiamengzhou in https://github.com/tatsu-lab/alpaca_eval/pull/370
* [ENH] add the code to compute instruction_following by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/371
* [ENH] adding simplified glm by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/372
* [BUG] backward compatibility vllm do_sample -> use_beam_search by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/373

New Contributors
* angelahzyuan made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/359
* sxjscience made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/367

**Full Changelog**: https://github.com/tatsu-lab/alpaca_eval/compare/v0.6.3...v0.6.4

What's Changed
* Add the evaluation result for our latest model by hendrydong in https://github.com/tatsu-lab/alpaca_eval/pull/286
* Add Ghost 7B Alpha to AlpacaEval by lh0x00 in https://github.com/tatsu-lab/alpaca_eval/pull/288
* Add link for FsfairX-Zephyr-Chat-v0.1 by hendrydong in https://github.com/tatsu-lab/alpaca_eval/pull/289
* add Qwen1.5-110B-Chat self-report results by Lukeming-tsinghua in https://github.com/tatsu-lab/alpaca_eval/pull/291
* [ENH] verifying all the qwens by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/292
* Enable analyzing evaluators/annotators on data without multiple generator models by rdnfn in https://github.com/tatsu-lab/alpaca_eval/pull/293
* Add Storm-7B to AlpacaEval by yifan123 in https://github.com/tatsu-lab/alpaca_eval/pull/294
* Use verified by default by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/297
* Add SPPO-Mistral7B-PairRM to AlpacaEval by Edward-Sun in https://github.com/tatsu-lab/alpaca_eval/pull/298
* Add ExPO results to AlpacaEval by chujiezheng in https://github.com/tatsu-lab/alpaca_eval/pull/299
* Fix typo in README.md by tongyx361 in https://github.com/tatsu-lab/alpaca_eval/pull/302
* Add Yi-Large Preview to AlpacaEval by HyperdriveHustle in https://github.com/tatsu-lab/alpaca_eval/pull/304
* "Add Mistral-7B+RAHF-DUAL+LoRA to AlpacaEval" by LiuAmber in https://github.com/tatsu-lab/alpaca_eval/pull/307
* [verified] Yi-large by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/309
* [ADD] GPT4-o by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/311
* [ENH] add LC SEM by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/317
* llama3 evaluator by zhuang-li in https://github.com/tatsu-lab/alpaca_eval/pull/314
* Update README.md by zhuang-li in https://github.com/tatsu-lab/alpaca_eval/pull/315
* [CLEAN] move evaluators lb llama3 by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/318
* [ENH] vicuna 1.5 by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/319
* Add Llama-3-Instruct-8B-SimPO to AlpacaEval by xiamengzhou in https://github.com/tatsu-lab/alpaca_eval/pull/320
* [ENH] Use multi threading instead of processing by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/321
* Add Aligner 2B+GPT-4 Turbo (04/09) Results by AlignInc in https://github.com/tatsu-lab/alpaca_eval/pull/324
* Add REBEL-Llama-3-8B-Instruct to AlpacaEval by ZhaolinGao in https://github.com/tatsu-lab/alpaca_eval/pull/326
* [ENH&BUG] improve VLLM by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/330
* Add ExPO + `Llama-3-Instruct-8B-SimPO` results by chujiezheng in https://github.com/tatsu-lab/alpaca_eval/pull/331
* fix model link by chujiezheng in https://github.com/tatsu-lab/alpaca_eval/pull/332
* Add merlinite-7B-AOT to AlpacaEval by imelnyk in https://github.com/tatsu-lab/alpaca_eval/pull/334
* [BUG] fix bs in VLLM and add chatml by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/338
* Add Together-MoA, Together-MoA-Lite to AlpacaEval by IsThatYou in https://github.com/tatsu-lab/alpaca_eval/pull/342
* Add Nanbeige2-16B-Chat to AlpacaEval by yuani114 in https://github.com/tatsu-lab/alpaca_eval/pull/345
* Add claude-3-5-sonnet-20240620 to AlpacaEval by MarjovanLier in https://github.com/tatsu-lab/alpaca_eval/pull/348
* [BUG] trust repo alpaca_eval by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/349
* Add OpenPipe Mixture of Agents model to Alpaca Eval by saum7800 in https://github.com/tatsu-lab/alpaca_eval/pull/347
* Add Storm-7B, Storm-7B (best-of-64) to AlpacaEval by yifan123 in https://github.com/tatsu-lab/alpaca_eval/pull/344
* Add Infinity-Instruct-3M-0613-Mistral-7B to AlpacaEval by cszhengyh in https://github.com/tatsu-lab/alpaca_eval/pull/351

New Contributors
* hendrydong made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/286
* lh0x00 made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/288
* yifan123 made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/294
* Edward-Sun made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/298
* chujiezheng made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/299
* tongyx361 made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/302
* LiuAmber made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/307
* zhuang-li made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/314
* xiamengzhou made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/320
* ZhaolinGao made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/326
* imelnyk made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/334
* IsThatYou made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/342
* MarjovanLier made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/348
* saum7800 made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/347
* cszhengyh made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/351

**Full Changelog**: https://github.com/tatsu-lab/alpaca_eval/compare/v0.6.2...v0.6.3

What's Changed
* [BUG] backward compatibility with AF by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/278
* Add Nanbeige-Plus-Chat-v0.1 to AlpacaEval by yuani114 in https://github.com/tatsu-lab/alpaca_eval/pull/279
* Update README.md by Dominic789654 in https://github.com/tatsu-lab/alpaca_eval/pull/280
* [BUG] revert to GPT4 preview 1106 by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/283
* Add support for analyzing evaluators with custom cross-annotations by rdnfn in https://github.com/tatsu-lab/alpaca_eval/pull/281
* [ENH] llama3 by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/285

New Contributors
* Dominic789654 made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/280
* rdnfn made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/281

**Full Changelog**: https://github.com/tatsu-lab/alpaca_eval/compare/v0.6.1...v0.6.2

What's Changed
* Add Aligner-2B+Qwen1.5-72B-Chat & Aligner-2B+Claude3 Opus to AlpacaEval by AlignInc in https://github.com/tatsu-lab/alpaca_eval/pull/259
* Supplement for Aligner by AlignInc in https://github.com/tatsu-lab/alpaca_eval/pull/261
* Add Ein-70B-v0.1 to AlpacaEval by bin-bi in https://github.com/tatsu-lab/alpaca_eval/pull/262
* Add TempNet-LLaMA2-Chat to AlpacaEval by xumao-nju in https://github.com/tatsu-lab/alpaca_eval/pull/264
* Add Conifer-7B-DPO to AlpacaEval by liulixin29 in https://github.com/tatsu-lab/alpaca_eval/pull/267
* Updating link to a super fast demo! by kyleliang919 in https://github.com/tatsu-lab/alpaca_eval/pull/268
* Add Nanbeige2-8B-Chat to AlpacaEval by yuani114 in https://github.com/tatsu-lab/alpaca_eval/pull/274
* [ENH] adding drbx and gpt4 turbo by YannDubs in https://github.com/tatsu-lab/alpaca_eval/pull/275

New Contributors
* AlignInc made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/259
* bin-bi made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/262
* xumao-nju made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/264
* liulixin29 made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/267
* yuani114 made their first contribution in https://github.com/tatsu-lab/alpaca_eval/pull/274

**Full Changelog**: https://github.com/tatsu-lab/alpaca_eval/compare/v0.6...v0.6.1