Internlm

Highlights
* Support centos and ubuntu dockerfile
* Support runtime gpu flops and nccl allreduce speed test

What's Changed

🚀 Features
* Implement uniform_init for tensor by Pryest in https://github.com/InternLM/InternLM/pull/252
* Support centos and ubuntu dockerfile by li126com in https://github.com/InternLM/InternLM/pull/220 https://github.com/InternLM/InternLM/pull/243
* Support writer `add_scalars` for writing dict data by huangting4201 in https://github.com/InternLM/InternLM/pull/257
* Support runtime gpu flops and nccl allreduce speed test by sunpengsdu in https://github.com/InternLM/InternLM/pull/254


💥 Improvements
-

🐞 Bug fixes
* Fix StreamingDataset does not have an `len` method by 00INDEX in https://github.com/InternLM/InternLM/pull/251
* Fix argument missing in getting loss metrics by MagicDevilZhang in https://github.com/InternLM/InternLM/pull/256
* Fix the error that RotaryEmbedding is converted to a non-fp32 format during operation by YWMditto in https://github.com/InternLM/InternLM/pull/239

📚 Documentations
* Update readme structure by huangting4201 in https://github.com/InternLM/InternLM/pull/240
* Support readthedocs by huangting4201 in https://github.com/InternLM/InternLM/pull/245 https://github.com/InternLM/InternLM/pull/264


🌐 Other
-

Known issues
-

🌟 **Highlights**
- 🛠 **Data Contamination Analysis**: A novel feature for analyzing and ensuring the integrity of dataset inputs.
- 🧠 **Enhanced Subjective Evaluation**: Implementation of a new subjective judgement system, providing more nuanced and accurate evaluations.
- 🚀 **Chat Style Inferencer Support**: Introduction of a new chat style inferencer, enhancing interactive capabilities.
- 🌐 **Multilingual Features**: Expansion to support Chinese versions of commonsenseqa, crowspairs, and nq datasets.
- 📊 **New Datasets Integration**: Addition of wikibench, rolebench, and updated versions of gsm8k and MathBench datasets for broader research applications.
- 🛠 **Enhancements and Bug Fixes**: Numerous improvements including a new subjective judgement system and updates in MathBench CodeInterpreter.
- 📝 **Documentation and API Updates**: Comprehensive updates to README and API interfaces for better user guidance and experience.

🚀 **New Features & Enhancements**
- Support for chat style inferencer, offering a more dynamic interaction model (643).
- Addition of Chinese versions for key datasets: commonsenseqa, crowspairs, and nq (144).
- Introduction of the wikibench dataset, providing a new benchmark for knowledge-based tasks (655).
- Updated gsm8k and MathBench configurations for enhanced performance and accuracy (652, 657).
- Addition of rolebench dataset, expanding the range of evaluative scenarios (633).
- Implementation of new subjective judgement criteria for improved assessment accuracy (660).
- Integration of advanced models like qwen-1.8b/72b and deepseek-7b/67b in the platform's configuration (672).
- Launch of Data Contamination Analysis as a new feature, enhancing data integrity checks (639).

🛠 **Improvements & Fixes**
- Removal of colossalai dependency to streamline operations (645).
- Resolution of various bugs including hellaswag_ppl_47bff9 and standard deviation summarizer issues (648, 675).
- Update and fix of the MathBench CodeInterpreter and related bugs (657).
- Enhancement of API interface for improved functionality and user experience (681).

📚 **Documentation Updates**
- Updated README for clearer guidance and information (682).
- Documentation and docstring updates for accuracy and comprehensiveness (684).

🎊 **New Contributors**
- A warm welcome to new contributors rolellm, liyucheng09, and xmshi-trio. Your contributions have significantly enriched OpenCompass!

🔗 **Full Changelog**
* [Fix] remove colossalai dependency by yingfhu in https://github.com/open-compass/opencompass/pull/645
* [Fix] Fix hellaswag_ppl_47bff9 by Leymore in https://github.com/open-compass/opencompass/pull/648
* [Feature] Support chat style inferencer. by mzr1996 in https://github.com/open-compass/opencompass/pull/643
* [Feature] Add Chinese version: commonsenseqa, crowspairs and nq by liushz in https://github.com/open-compass/opencompass/pull/144
* [Feature] Add wikibench dataset by liushz in https://github.com/open-compass/opencompass/pull/655
* [Feat] update gsm8k and math agent config by yingfhu in https://github.com/open-compass/opencompass/pull/652
* [Feature] Update MathBench CodeInterpreter & fix MathBench Bug by liushz in https://github.com/open-compass/opencompass/pull/657
* added rolebench dataset. by rolellm in https://github.com/open-compass/opencompass/pull/633
* New subjective judgement by bittersweet1999 in https://github.com/open-compass/opencompass/pull/660
* [Feature] Add qwen-1.8b/72b and deepseek-7b/67b configs by Leymore in https://github.com/open-compass/opencompass/pull/672
* Add Data Contamination Analysis [New Feature] by liyucheng09 in https://github.com/open-compass/opencompass/pull/639
* [Fix] fix bug on standart_deviation summarizer by jingmingzhuo in https://github.com/open-compass/opencompass/pull/675
* update medbench by xmshi-trio in https://github.com/open-compass/opencompass/pull/678
* [Enhancement] Update API Interface by tonysy in https://github.com/open-compass/opencompass/pull/681
* [Doc] Update README by kennymckormick in https://github.com/open-compass/opencompass/pull/682
* [Feat] support pr merge test ci by yingfhu in https://github.com/open-compass/opencompass/pull/669
* [Feature] enhance the ability of humaneval_postprocess by jingmingzhuo in https://github.com/open-compass/opencompass/pull/676
* [Sync] Update codes by yingfhu in https://github.com/open-compass/opencompass/pull/683
* [Docs] fix docstring by yingfhu in https://github.com/open-compass/opencompass/pull/684
* new version of subject by bittersweet1999 in https://github.com/open-compass/opencompass/pull/680
* fixed small problem of new version subject evaluation by bittersweet1999 in https://github.com/open-compass/opencompass/pull/686
* [Sync] bump version to 0.2.0 by yingfhu in https://github.com/open-compass/opencompass/pull/690


- Explore the detailed changes and contributions in the full changelog: [OpenCompass Changelog](https://github.com/open-compass/opencompass/compare/0.1.9...0.2.0).

Thank you to all contributors for your hard work and dedication. OpenCompass v0.2.0 marks another step forward in our journey, bringing enhanced features and capabilities to the community. Let's continue to innovate and expand the horizons of OpenCompass! 🎉🌐💡

🌟 Highlights
- 🚀 **New API Integrations:** A leap forward with the addition of multiple new APIs, including Baidu, Moonshot, Sensetime, and more, broadening the scope and capabilities of OpenCompass.
- 🔵 **Circular Evaluation Feature:** Introducing Circular Eval, an enhancement for comprehensive and dynamic evaluations within the platform.
- 🤖 Turbomind Inference Integration: Integration of Turbomind inference through its RPC API, enhancing the platform's inferencing capabilities.


🚀 New Features & Enhancements
- **Model & API Development:** Explore new capabilities with DataCanvas Alaya LM, Lightllm API, 360API, and enhanced Turbomind Python API integration ([612](https://github.com/open-compass/opencompass/pull/612), [#613](https://github.com/open-compass/opencompass/pull/613), [#601](https://github.com/open-compass/opencompass/pull/601), [#484](https://github.com/open-compass/opencompass/pull/484)).
- **Circular Evaluation Implementation:** Elevate your evaluation methods with the newly added Circular Eval feature, offering a more nuanced and detailed analysis capability ([610](https://github.com/open-compass/opencompass/pull/610)).
- **Rich Dataset Additions:** Enrich your research with new datasets - FinanceIQ, SVAMP, GSM_Hard, and updated Mathbench for diverse applications ([596](https://github.com/open-compass/opencompass/pull/596), [#604](https://github.com/open-compass/opencompass/pull/604), [#619](https://github.com/open-compass/opencompass/pull/619), [#580](https://github.com/open-compass/opencompass/pull/580), [#607](https://github.com/open-compass/opencompass/pull/607)).

🛠 Improvements & Fixes
- **Subjective Evaluation Bug Fixes:** Improved accuracy in subjective evaluations ([589](https://github.com/open-compass/opencompass/pull/589)).
- **Dataset and Feature Fixes:** Resolving issues in CMB dataset, various feature enhancements, and fixes ([587](https://github.com/open-compass/opencompass/pull/587), [#592](https://github.com/open-compass/opencompass/pull/592), [#615](https://github.com/open-compass/opencompass/pull/615), [#632](https://github.com/open-compass/opencompass/pull/632)).

📚 Documentation Updates
- **README & FAQ Enhancements:** Updated for better clarity and assistance ([582](https://github.com/open-compass/opencompass/pull/582), [#622](https://github.com/open-compass/opencompass/pull/622), [#628](https://github.com/open-compass/opencompass/pull/628), [#629](https://github.com/open-compass/opencompass/pull/629)).
- **Typo and Spelling Corrections:** Ensuring accuracy and professionalism in documentation ([594](https://github.com/open-compass/opencompass/pull/594), [#637](https://github.com/open-compass/opencompass/pull/637)).

🎊 New Contributors
Welcoming new contributors to the OpenCompass family!
- **rahidzeynal**, **Sniper970119**, **ZhangRaymond**, **HunterKruger**, **helloyongyang**, and **Yggdrasill7D6**. Your contributions are greatly appreciated!

What's Changed
* Add author as: author='OpenCompass Contributors' by rahidzeynal in https://github.com/open-compass/opencompass/pull/578
* [Doc] Update README by tonysy in https://github.com/open-compass/opencompass/pull/582
* [Feature] Update mathbench by tonysy in https://github.com/open-compass/opencompass/pull/580
* Fix bugs in subjective evaluation by frankweijue in https://github.com/open-compass/opencompass/pull/589
* [Fix] fix cmb dataset by Leymore in https://github.com/open-compass/opencompass/pull/587
* [Fix] change save_every defaults to 1 by yingfhu in https://github.com/open-compass/opencompass/pull/592
* update word spell by Sniper970119 in https://github.com/open-compass/opencompass/pull/594
* Add FinanceIQ dataset by ZhangRaymond in https://github.com/open-compass/opencompass/pull/596
* [Feat] support humaneval and mbpp passk by yingfhu in https://github.com/open-compass/opencompass/pull/598
* [Feature] Add multi-prompt generation demo by jingmingzhuo in https://github.com/open-compass/opencompass/pull/568
* Mathbench update postprocess by liushz in https://github.com/open-compass/opencompass/pull/600
* [Feature] Add arithmetic to mathbench by liushz in https://github.com/open-compass/opencompass/pull/607
* Add support for DataCanvas Alaya LM by HunterKruger in https://github.com/open-compass/opencompass/pull/612
* [Feature] Support Lightllm api by helloyongyang in https://github.com/open-compass/opencompass/pull/613
* [Feature] Support 360API and FixKRetriever for CSQA dataset by tonysy in https://github.com/open-compass/opencompass/pull/601
* Integrate turbomind python api by lvhan028 in https://github.com/open-compass/opencompass/pull/484
* [Bug] Update api with generation_kargs by tonysy in https://github.com/open-compass/opencompass/pull/614
* [Fix] Fix gen inferencer by Leymore in https://github.com/open-compass/opencompass/pull/615
* [Docs] update ds1000 code eval docs by yingfhu in https://github.com/open-compass/opencompass/pull/618
* [Feature] Add SVAMP dataset by liushz in https://github.com/open-compass/opencompass/pull/604
* [Feature] support download from modelscope by KevinNuNu in https://github.com/open-compass/opencompass/pull/534
* [Doc] Update README and requirements. by tonysy in https://github.com/open-compass/opencompass/pull/622
* [Sync] Fix cmnli, fix vicuna meta template, fix longbench postprocess and other minor fixes by Leymore in https://github.com/open-compass/opencompass/pull/625
* [API] Update API by tonysy in https://github.com/open-compass/opencompass/pull/624
* [Feature] Add circular eval by Leymore in https://github.com/open-compass/opencompass/pull/610
* [Doc] Update FAQ by Leymore in https://github.com/open-compass/opencompass/pull/628
* [Doc] Update README by tonysy in https://github.com/open-compass/opencompass/pull/629
* [Bug] fix icl eval with nested list by yingfhu in https://github.com/open-compass/opencompass/pull/632
* Fix LightllmAPI list bug by helloyongyang in https://github.com/open-compass/opencompass/pull/635
* fix typo in README by Yggdrasill7D6 in https://github.com/open-compass/opencompass/pull/637
* [Sync] update codes by Leymore in https://github.com/open-compass/opencompass/pull/641
* [Feature] Add GSM_Hard dataset by liushz in https://github.com/open-compass/opencompass/pull/619
* [Feat] support zhipu post process by yingfhu in https://github.com/open-compass/opencompass/pull/642
* [Sync] Bump version to 0.1.9 by Leymore in https://github.com/open-compass/opencompass/pull/644

Explore the detailed changes in the [full changelog](https://github.com/open-compass/opencompass/compare/0.1.8...0.1.9).

Thank you to all the contributors for this release. Your dedication and hard work continue to enhance OpenCompass, making it an ever-evolving and dynamic tool for the community. Let's dive into the new possibilities with OpenCompass v0.1.9! 🎉🧮💻

🔥 Highlights
- 🌐 **New Dataset Integrations:** Expanding our dataset collection with Tabmwp, py150, maxmin, and more.
- 💡 **Compatibility and API Support:** Enhancements with MiniGPT-4 and MiniMax API, and support for Xunfei API.
- 🛠️ **Local Environment and Debugging Improvements:** Streamlined local debugging and usage of datasets from local paths.

🚀 New Features & Enhancements
- **Datasets Galore:** Unleash the power of new datasets including Tabmwp, py150, maxmin, and updates to existing ones like Mathbench for broader research scope ([505](https://github.com/open-compass/opencompass/pull/505), [#546](https://github.com/open-compass/opencompass/pull/546), [#562](https://github.com/open-compass/opencompass/pull/562)).
- **MiniGPT-4 & MiniMax API Compatibility:** Stay up-to-date with the latest versions and extended API support ([539](https://github.com/open-compass/opencompass/pull/539), [#548](https://github.com/open-compass/opencompass/pull/548)).
- **Xunfei API Model & Update:** Explore new possibilities with the integration and update of Xunfei API ([547](https://github.com/open-compass/opencompass/pull/547), [#572](https://github.com/open-compass/opencompass/pull/572)).

🛠 Improvements & Fixes
- **Local Debug Mode Restriction:** Enhanced resource management in local debug mode ([522](https://github.com/open-compass/opencompass/pull/522) by yingfhu).
- **Various Fixes and Updates:** Addressing typos, import issues, and log redirections for smoother operation ([520](https://github.com/open-compass/opencompass/pull/520), [#549](https://github.com/open-compass/opencompass/pull/549), [#551](https://github.com/open-compass/opencompass/pull/551), [#555](https://github.com/open-compass/opencompass/pull/555), [#564](https://github.com/open-compass/opencompass/pull/564)).

📚 Documentation Updates
- **Enhanced README and FAQs:** Get all your queries answered and understand OpenCompass better with updated documentation ([523](https://github.com/open-compass/opencompass/pull/523), [#531](https://github.com/open-compass/opencompass/pull/531), [#535](https://github.com/open-compass/opencompass/pull/535), [#540](https://github.com/open-compass/opencompass/pull/540), [#567](https://github.com/open-compass/opencompass/pull/567)).
- **Typo Corrections:** Ensuring clarity and accuracy in our documentation ([530](https://github.com/open-compass/opencompass/pull/530), [#533](https://github.com/open-compass/opencompass/pull/533)).

🎊 New Contributors
A warm welcome to the new members of the OpenCompass community!
- **Sanster**, **ayushrakesh**, **HimanshuMahto**, **shresthasurav**, **bittersweet1999**, and **jingmingzhuo**. Thank you for your valuable contributions!

Changelog

* add multi model viz by Sanster in https://github.com/open-compass/opencompass/pull/509
* fix typo in WSC prompt by Sanster in https://github.com/open-compass/opencompass/pull/520
* [Fix] fix local debug mode not restrict the resources by yingfhu in https://github.com/open-compass/opencompass/pull/522
* Update README.md - one enhancement. by ayushrakesh in https://github.com/open-compass/opencompass/pull/523
* Typo error in README.md by HimanshuMahto in https://github.com/open-compass/opencompass/pull/531
* docs: fix typos in markdown files by shresthasurav in https://github.com/open-compass/opencompass/pull/530
* [Doc] Update README and FAQ by tonysy in https://github.com/open-compass/opencompass/pull/535
* [fFeat] Add an opensource dataset Tabmwp by bittersweet1999 in https://github.com/open-compass/opencompass/pull/505
* [Feature]: To be compatible with the latest version of MiniGPT-4 by YuanLiuuuuuu in https://github.com/open-compass/opencompass/pull/539
* [Doc] Update README by tonysy in https://github.com/open-compass/opencompass/pull/540
* [Feat] support xunfei api model by yingfhu in https://github.com/open-compass/opencompass/pull/547
* [Feature] Add support for MiniMax API by tonysy in https://github.com/open-compass/opencompass/pull/548
* 【Feature】Update Mathbench dataset prompt and fix small errors by liushz in https://github.com/open-compass/opencompass/pull/546
* [Fix] fix filename typo by yingfhu in https://github.com/open-compass/opencompass/pull/549
* [Feat] support cidataset by yingfhu in https://github.com/open-compass/opencompass/pull/538
* [Fix] fix registry error with internal by yingfhu in https://github.com/open-compass/opencompass/pull/551
* [Fix] fix unnecessary import and update requirements by yingfhu in https://github.com/open-compass/opencompass/pull/555
* [Fix] fix log re-direct by yingfhu in https://github.com/open-compass/opencompass/pull/564
* Add py150 and maxmin by jingmingzhuo in https://github.com/open-compass/opencompass/pull/562
* [Doc] Update api.txt by tonysy in https://github.com/open-compass/opencompass/pull/567
* [Docs] add humanevalx dataset link in config by yingfhu in https://github.com/open-compass/opencompass/pull/559
* [Docs] fix GLUE_CoLA dataset name error by KevinNuNu in https://github.com/open-compass/opencompass/pull/533
* [Feature] Update xunfei api by tonysy in https://github.com/open-compass/opencompass/pull/572
* [Feature] Add CMB zero-shot evaluation by Leymore in https://github.com/open-compass/opencompass/pull/571
* [Feature] Use dataset in local path by Leymore in https://github.com/open-compass/opencompass/pull/570
* [Sync] update model configs by Leymore in https://github.com/open-compass/opencompass/pull/574
* [Sync] Bump version to 0.1.8 by Leymore in https://github.com/open-compass/opencompass/pull/576

Explore the detailed changes in the [full changelog](https://github.com/open-compass/opencompass/compare/0.1.7...0.1.8).

Thank you to everyone who contributed to this release. Your efforts are immensely appreciated and are helping to make OpenCompass a more robust and versatile tool. Let's continue to push the boundaries with OpenCompass v0.1.8! 🚀🌐🛠️

Provide with more parsed datasets:

`OpenCompassData-core.zip`

| AGIEval | ARC | BBH | ceval | CLUE | cmmlu |
|:----------------|:-----------------|:-----------------|:----------------|:----------------|:----------------|
| commonsenseqa | drop | FewCLUE | flores_first100| GAOKAO-BENCH | gsm8k |
| hellaswag | humaneval | lambada | LCSTS | math | mbpp |
| mmlu | nq | openbookqa | piqa | race | siqa |
| strategyqa | summedits | SuperGLUE | TheoremQA | triviaqa | tydiqa |
| winogrande | xstory_cloze | Xsum | | | |

`OpenCompassData-complete.zip`

| AGIEval | anli | ARC | BBH | ceval | cleva |
|:--------------------|:-------------------|:-------------------|:-------------------|:-------------------|:-------------------|
| CLUE | CMB | cmmlu | commonsenseqa | drop | ds1000 |
| FewCLUE | flores200_dataset | flores_first100 | game24 | GAOKAO-BENCH | govrep |
| gsm8k | hellaswag | humaneval | jigsawmultilingual| lambada | lawbench |
| LCSTS | math | mbpp | mmlu | narrativeqa | nq |
| openbookqa | piqa | QASPER | race | realtoxicprompts | scibench |
| siqa | SQuAD2.0 | strategyqa | summedits | SummScreen | SuperGLUE |
| TheoremQA | triviaqa | triviaqa-rc | tydiqa | winogrande | xiezhi |
| xlsum | xstory_cloze | Xsum | FinanceIQ | | |