Modelgauge

What's Changed

* Tests and SUTs now have a member variable UID, which gets passed into their constructor.
* Introduced `newhelm_test` and `newhelm_sut` decorators to give us better hooks into user code.
* New command `list-suts` to tell you what secrets each SUT uses.
* Bug fixes for SafeTest, max_test_items, our integration with Together

New Contributors
* 0xkerem made their first contribution in https://github.com/mlcommons/newhelm/pull/263

**Full Changelog**: https://github.com/mlcommons/newhelm/compare/v0.2.3...v0.2.4

What's Changed

* The results from a test in `TestRecord` switched from `List[Result]` to a test specific `TypedData`. This allows Tests to report their results in a more natural structured form, as well as provide documentation on what that form is.
* More SAFE tests, including `benign` tests.

**Full Changelog**: https://github.com/mlcommons/newhelm/compare/v0.2.2...v0.2.3

What's Changed

* Many tutorials added for SUTs, secrets, and options.
* Various bug fixes and clean upl

**Full Changelog**: https://github.com/mlcommons/newhelm/compare/v0.2.1...v0.2.2

What's Changed

* Added "Safe AI with Flexible Evaluation" Tests (SafeTest) for three hazard types.
* We now include a `source_id` in PromptWithContext, linking that prompt back to the ID used in the original data source.
* Reworked how Secrets work internally, making their use much more explicit.
* You can now see all possible secrets with `list-secrets` command.
* Lots of error messaging improvements, including:
* If you are missing any secrets necessary a run, you are notified immediately. Error message provides guidance for how to add them.
* TogetherAI provides more detailed error messages.
* When exceptions happen in SUTs, report what the SUT was working on.
* HuggingFaceSUT can now use [Gated Models](https://huggingface.co/docs/hub/en/models-gated)

**Full Changelog**: https://github.com/mlcommons/newhelm/compare/v0.2.0...v0.2.1

What's Changed
* The secrets file moved from `secrets/default.json` to `config/secrets.toml` to try and make hand writing that file easier.
* Added XSTest
* Annotator responses are now cached, meaning you can redo a full run only hitting the cache. You can disable caching with `--no-caching`.
* The TestRecord is written to an output json file instead of just to the screen.

Fixing package relationships so that the correct plugins are installed.