What's Changed
* pip package by pufanyi in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/1
* Fix mmbench dataset submission format by pufanyi in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/7
* [Feat] add correct tensor parallelism for larger size model. by Luodian in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/4
* update version to 0.1.1 by pufanyi in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/9
* [Tasks] Fix MMBench by pufanyi in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/13
* [Fix] Fix llava reproduce error by kcz358 in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/24
* add_ocrbench by echo840 in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/28
* Joshua/olympiadbench by JvThunder in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/37
* [WIP] adding mmbench dev evaluation (75) by Luodian in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/46
* Add `llava` model for 🤗 Transformers by lewtun in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/47
* Fix types to allow nullables in `llava_hf.py` by lewtun in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/55
* Add REC tasks for testing model ability to locally ground objects, given a description. This adds REC for all RefCOCO datasets. by hunterheiden in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/52
* [Benchmarks] RealWorldQA by pufanyi in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/57
* add Llava-SGlang by jzhang38 in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/54
* Add MathVerse by CaraJ7 in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/60
* Fix typo in Qwen-VL that was causing "reference before assignment" by tupini07 in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/61
* New Task: ScreenSpot - Grounding (REC) and instruction generation (REG) on screens by hunterheiden in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/63
* [New Task] WebSRC (multimodal Q&A on web screenshots) by hunterheiden in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/69
* Bugfix: WebSRC should be token-level F1 NOT character-level by hunterheiden in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/70
* Multilingual LLava bench by gagan3012 in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/56
* [Fix] repr llava doc by cocoshe in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/36
* add idefics2 by jzhang38 in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/59
* [Feat] Add qwen vl api by kcz358 in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/73
* Adding microsoft/Phi-3-vision-128k-instruct model. by vfragoso in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/87
* Add MathVerse in README.md by CaraJ7 in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/97
* add MM-UPD by AtsuMiyai in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/95
* add Conbench by Gumpest in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/100
* Update conbench in README by Gumpest in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/101
* update gpt-3.5-turbo version by AtsuMiyai in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/107
* [Upgrade to v0.2] Embracing Video Evaluations with LMMs-Eval by Luodian in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/108
New Contributors
* pufanyi made their first contribution in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/1
* Luodian made their first contribution in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/4
* kcz358 made their first contribution in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/24
* echo840 made their first contribution in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/28
* JvThunder made their first contribution in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/37
* lewtun made their first contribution in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/47
* hunterheiden made their first contribution in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/52
* jzhang38 made their first contribution in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/54
* CaraJ7 made their first contribution in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/60
* tupini07 made their first contribution in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/61
* gagan3012 made their first contribution in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/56
* cocoshe made their first contribution in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/36
* vfragoso made their first contribution in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/87
* AtsuMiyai made their first contribution in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/95
* Gumpest made their first contribution in https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/100
**Full Changelog**: https://github.com/EvolvingLMMs-Lab/lmms-eval/compare/v0.1.0...v0.2.0