Mallm

What's Changed
* Aqua rat by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/145
* fix fstring by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/146
* toml PEP 621 complicance + ifeval by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/147
* Judge Agent by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/148


**Full Changelog**: https://github.com/Multi-Agent-LLMs/mallm/compare/v1.0.3...v1.0.4

**Full Changelog**: https://github.com/Multi-Agent-LLMs/mallm/compare/v1.0.2...v1.0.3

**Full Changelog**: https://github.com/Multi-Agent-LLMs/mallm/compare/v1.0.1...v1.0.2

What's Changed
* ifeval evaluation (instruction following) by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/139
* no duplicate answer choices by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/143
* Feat/challenge results by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/144


**Full Changelog**: https://github.com/Multi-Agent-LLMs/mallm/compare/v1.0.0...v1.0.1

Added release on pypi

What's Changed
* Create update_readme.yml by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/72
* add workflow py script by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/73
* Added QA metric for GPQA, MMLU, etc. by ItsNiklas in https://github.com/Multi-Agent-LLMs/mallm/pull/74
* Small Code Style Improvements by ItsNiklas in https://github.com/Multi-Agent-LLMs/mallm/pull/77
* Fix readme updater by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/78
* feat: added batch executor for mallm by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/81
* ResponseGenerators: Handling prompts, extraction, and agreements by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/79
* More robust multichoice metric + fixed datasets by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/82
* Prompt improvements (+ majority consensus fix) by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/85
* Fix/no unlimited voting by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/88
* Extensive evaluation by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/92
* 1) sort output file after finishing 2) comparable moderator by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/91
* feat: When there is a dataset issue, print exactly what the issue is by jpwahle in https://github.com/Multi-Agent-LLMs/mallm/pull/98
* Support hf datasets by jpwahle in https://github.com/Multi-Agent-LLMs/mallm/pull/93
* fix: Fix a bug where forgetting trailing slash in memory bucket leads to undesired behaviour by jpwahle in https://github.com/Multi-Agent-LLMs/mallm/pull/100
* Ablation by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/99
* Refactor/evaluator by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/90
* fix: Fix a bug where the HF dataset is not sorted properly. by jpwahle in https://github.com/Multi-Agent-LLMs/mallm/pull/102
* Fix/data load and debug output by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/105
* Feat/plotting by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/106
* squad metric enhancements by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/107
* Paraphrase types agent generator by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/103
* fix out full path by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/109
* Feat/rich output by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/110
* fix: unanimity is unreachable because of faulty condition by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/108
* Prompt changes and minor adjustments by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/114
* Distinct-N metric by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/116
* Remove dbm memory by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/115
* feat: added instruction templates by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/118
* batch executor refinements by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/113
* More expressive param names + flexible number of neutral agents by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/119
* feat: all agents generate a first draft and after that improve by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/120
* feat: added mmlu pro dataset by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/123
* feat: add musr dataset by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/125
* feat: add math lvl 5 dataset downloader by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/126
* Freely combine persona types + NoPersonaGenerator by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/122
* feat: added prompts for new datasets by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/127
* feat: add mallm command line scripts by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/128
* Feat/evaluator with alterations by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/129
* feat: added metric that checks if answer is included in response by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/131
* feat: added discord webhook to batch processing by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/130
* feat: add mmlu dataset by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/132
* BBQ, MoCa, MoralExceptQA Datasets + metadata field by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/133
* Feat/summarize decision protocol by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/124
* policy feedback agent by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/134
* Persona diversity index by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/136
* WinoGrande, ETHICS datasets by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/137
* Consensus Voting by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/138
* feat: add new commandline script to execute mallm batch mode by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/140
* feat: add challenge of final answer to test consistency by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/141
* update readme by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/142

New Contributors
* jpwahle made their first contribution in https://github.com/Multi-Agent-LLMs/mallm/pull/98

**Full Changelog**: https://github.com/Multi-Agent-LLMs/mallm/compare/v0.1.0-alpha...v1.0.0

This is the first release of MALLM (alpha).

What's Changed
* Fix/setup by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/1
* memory redesign, more extensive output logs, refactoring, bug fixes by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/2
* Feat/new build system by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/3
* refactor: add abstract class for datasets by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/4
* feat: improve readme by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/6
* Feat/logging by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/7
* Feat/formatting by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/8
* Added GPQA Dataset, cffi dependency for linux by ItsNiklas in https://github.com/Multi-Agent-LLMs/mallm/pull/10
* Tgi implementation by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/9
* reorganized files by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/16
* Multi source datasets by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/19
* Create unit test by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/21
* Fix/discussion by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/17
* Refactor/coordinator by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/18
* installable package by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/25
* small fixes by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/26
* WMT119, paraphrase types, and fixes by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/28
* Feat/btvote by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/27
* fix etpc and context by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/35
* fix json stringify by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/36
* Feat/decision protocol by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/30
* feat: added agent history as a chat format by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/38
* samples left logging and type hints by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/37
* stream answers to reduce memory usage by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/40
* Strict types by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/42
* Refactor/GitHub action by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/45
* Openai support by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/47
* feat: added tests for coordinator by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/46
* Refactor/moderator by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/44
* Evaluation framework by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/50
* Feat: fixed number of turns by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/53
* Improve prompts by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/55
* Feat baseline by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/56
* Added IPIP Persona Generator by ItsNiklas in https://github.com/Multi-Agent-LLMs/mallm/pull/59
* fix missing hf_token, btvote not shuffled, max_samples by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/57
* Discussion length by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/58
* add pytest pre commit hook by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/60
* Added more decision protocols by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/49
* Split agree and answer by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/54
* refactor: introduce config dataclass to remove duplicate code and mak… by lkaesberg in https://github.com/Multi-Agent-LLMs/mallm/pull/64
* Improve extraction reliability by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/63
* failed samples logging by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/65
* Stability fixes by jonas-becker in https://github.com/Multi-Agent-LLMs/mallm/pull/66

New Contributors
* lkaesberg made their first contribution in https://github.com/Multi-Agent-LLMs/mallm/pull/1
* jonas-becker made their first contribution in https://github.com/Multi-Agent-LLMs/mallm/pull/2

**Full Changelog**: https://github.com/Multi-Agent-LLMs/mallm/commits/v0.1.0-alpha