Modelgauge

What's Changed
* Updated docs
* SafeTest compatible with python 3.11+
* Add new [Llama Guard 2](https://llama.meta.com/docs/model-cards-and-prompt-formats/meta-llama-guard-2) to `LlamaGuardAnnotator`
* Can configure `LlamaGuardAnnotator` with optional `llama_guard_version` parameter. Defaults to Llama Guard 2
* Minor changes to prompt/category formatting for Llama Guard 1. This may affect results.
* SafeTest can also be configured to use Llama Guard 1 or 2 as it's annotator. Defaults to version 2.


**Full Changelog**: https://github.com/mlcommons/modelgauge/compare/v0.5.0...v0.5.1

What's Changed

* Renamed to ModelGauge and started pushing to PyPI!
* A whole bunch of cleanups and preparation for the more public release.
* Caching now supports dicts.
* Unit tests to ensure you can install from PyPI and run in a notebook.
* Expand range of supported python versions to 3.10 and up.
* Remove benign hazard from SafeTest.
* Start setting up ReadTheDocs.


**Full Changelog**: https://github.com/mlcommons/modelgauge/compare/v0.3.3...v0.5.0

What's Changed
* Change SafeTest to data_april04 release.
* More prompts
* Removed safe-ben

**Full Changelog**: https://github.com/mlcommons/newhelm/compare/v0.3.2...v0.3.3

What's Changed
* `max_test_items` returns a relatively stable set of prompts
* Loading bar for plugins
* Have `list` command report prettier values for secrets
* Time out requests stuck on TogetherAI
* Updated docs
* Move `simple_test_runner` out of plugins and into core library

**Full Changelog**: https://github.com/mlcommons/newhelm/compare/v0.3.1...v0.3.2

What's Changed
* Fix bad version specification for `together` dependency, which was causing 0.3.0 to not actually install.
* Add Deepseek model that is now available on Together.
* Stabilize the order of TestItems in SafeTest to better utilize caching.

**Full Changelog**: https://github.com/mlcommons/newhelm/compare/v0.3.0...v0.3.1

What's Changed

* Reorganized the `run_data` folder and made several improvements to caching. **This breaks backward comparability**. Old files should just be ignored, but if you run into issues, probably best to just delete your `run_data` folder.
* Updated SafeTest to 02apr2024.
* We now have all SUTs in the [requested set](https://docs.google.com/document/d/11HsLhVFPsiwcwWIsou275u1HHbp8ZM8vkUCTjAcqLXE/edit), minus Deepseek.
* Simplified the command line to be `newhelm` once installed or `poetry run newhelm` when using the local repo.
* Annotations are now recorded per completion instead of per TestItem.
* HuggingFace sets pad token to default, which should remove warning messages.
* Added some enforcement of SUTCapabilities to help them be accurate.
* Remove all "Base" prefixes except BaseTest.

**Full Changelog**: https://github.com/mlcommons/newhelm/compare/v0.2.6...v0.3.0