Nnbench

This release contains the `Memo` class for better memory efficiency when benchmarking multiple models or datasets with large memory footprint in succession.

It also introduces `Transform`s as a way to encapsulate processing of records for multiple purposes, like generating statistics or serialization/deserialization.

What's Changed
* Example Guide: Benchmark on Saved Models by maxmynter in https://github.com/aai-institute/nnbench/pull/92
* Add a lightning demo to the README.md by nicholasjng in https://github.com/aai-institute/nnbench/pull/98
* Hotfix: Adjust snippet paths by nicholasjng in https://github.com/aai-institute/nnbench/pull/102
* Add Parameters to Benchmarks by maxmynter in https://github.com/aai-institute/nnbench/pull/103
* Run pre-commit autoupdate, bump dependencies by nicholasjng in https://github.com/aai-institute/nnbench/pull/104
* Implement a FSspec based ArtifactLoader by maxmynter in https://github.com/aai-institute/nnbench/pull/100
* Add Artifacts guide by maxmynter in https://github.com/aai-institute/nnbench/pull/107
* Remove the `ArtifactCollection` class by maxmynter in https://github.com/aai-institute/nnbench/pull/109
* Inline the `_cleanup` function for the `FilePathArtifactLoader`. by maxmynter in https://github.com/aai-institute/nnbench/pull/111
* Unique keys by maxmynter in https://github.com/aai-institute/nnbench/pull/112
* `functools.partial`ize the `nnbench.parametrize` decorators. by maxmynter in https://github.com/aai-institute/nnbench/pull/116
* Add transform proposal by nicholasjng in https://github.com/aai-institute/nnbench/pull/113
* Make `Transform` an abstract base class, delete iapply method ... by nicholasjng in https://github.com/aai-institute/nnbench/pull/118
* Check loaded module names before importing a new one from a file by nicholasjng in https://github.com/aai-institute/nnbench/pull/119
* Rollforward of lakefs-spec doc changes by nicholasjng in https://github.com/aai-institute/nnbench/pull/123
* Add transform submodule, parameter compression transform by nicholasjng in https://github.com/aai-institute/nnbench/pull/124
* Feature: Thunks as an alternative to artifacts by nicholasjng in https://github.com/aai-institute/nnbench/pull/120
* Add new memoization guide by nicholasjng in https://github.com/aai-institute/nnbench/pull/129
* State injection into setup and teardown by maxmynter in https://github.com/aai-institute/nnbench/pull/127
* Add global memo cache and integrate with the setUp and teardown injection by maxmynter in https://github.com/aai-institute/nnbench/pull/130
* Update dev requirements, add Memray artifacts to gitignore by nicholasjng in https://github.com/aai-institute/nnbench/pull/136
* Organize `nnbench.types.types` into different files by maxmynter in https://github.com/aai-institute/nnbench/pull/135
* Implement memo garbage collection by nicholasjng in https://github.com/aai-institute/nnbench/pull/137
* Update HuggingFace example to use memoization by nicholasjng in https://github.com/aai-institute/nnbench/pull/133


**Full Changelog**: https://github.com/aai-institute/nnbench/compare/v0.2.0...v0.3.0

Summary

This release makes a wide variety of benchmarking functionality available: Contexts for easier metadata organization, providers for basic information, file IO for record persistence on disk, and more.

It also adds new guides and code examples on how to benchmark code easily in a variety of scenarios, such as in orchestrator workflows and with duckDB.

What's Changed
* Add git info and Python package info providers by nicholasjng in https://github.com/aai-institute/nnbench/pull/64
* Group context structs under top-level semantic keys by nicholasjng in https://github.com/aai-institute/nnbench/pull/65
* Simplify nullcols helper, flatten nested context by nicholasjng in https://github.com/aai-institute/nnbench/pull/68
* Fix runner collection by checking file path first by nicholasjng in https://github.com/aai-institute/nnbench/pull/70
* Change `nnbench.reporter` file to submodule, add base reporter by nicholasjng in https://github.com/aai-institute/nnbench/pull/71
* Inline console reporter into BenchmarkReporter.display() by nicholasjng in https://github.com/aai-institute/nnbench/pull/76
* Orchestrator examples by maxmynter in https://github.com/aai-institute/nnbench/pull/73
* Improve some aspects on benchmark reporters by nicholasjng in https://github.com/aai-institute/nnbench/pull/77
* Add barebones duckDB reporter by nicholasjng in https://github.com/aai-institute/nnbench/pull/75
* Add BenchmarkContext Class by maxmynter in https://github.com/aai-institute/nnbench/pull/79
* Improve file drivers, add dict roundtrip methods to `BenchmarkRecord` by nicholasjng in https://github.com/aai-institute/nnbench/pull/82
* Unify batching in file drivers, remove read/write stubs on base reporter by nicholasjng in https://github.com/aai-institute/nnbench/pull/83
* Add compression algorithms facility to file IO by nicholasjng in https://github.com/aai-institute/nnbench/pull/84
* Streamlit Integration by maxmynter in https://github.com/aai-institute/nnbench/pull/81
* Add BQ example to showcase cloud streaming of benchmark records by nicholasjng in https://github.com/aai-institute/nnbench/pull/85
* Add parquet, ndjson drivers, change registration mechanisms by nicholasjng in https://github.com/aai-institute/nnbench/pull/87
* Revert file extension coercion, add raise statement for missing driver by nicholasjng in https://github.com/aai-institute/nnbench/pull/88
* Switch author and maintainer description in example .toml's by maxmynter in https://github.com/aai-institute/nnbench/pull/89
* Restructure test suite and add file IO roundtrip tests by nicholasjng in https://github.com/aai-institute/nnbench/pull/90
* Add duckDB example by nicholasjng in https://github.com/aai-institute/nnbench/pull/91
* Run `pre-commit autoupdate`, upgrade requirement lockfiles by nicholasjng in https://github.com/aai-institute/nnbench/pull/93


**Full Changelog**: https://github.com/aai-institute/nnbench/compare/v0.1.0...v0.2.0

This marks the first stable release of nnbench. From now on, the package is available for installation on PyPI using standard Python dependency management tools like pip and poetry.

Contents

* Definition and parametrization of benchmarks via decorators
* Collection and running in a benchmark runner class
* Result reporting via a benchmark reporter class (currently, only tabular console output is supported.)
* Documentation including quickstart, a more advanced example, and an API reference.

Breaking changes
None.

Deprecations
None.

This is a test PyPI release.