Autogluon.bench

We're excited to announce the release of AutoGluon-Bench 0.2.2!

For more details, please check out the [README](https://github.com/autogluon/autogluon-bench/blob/master/README.md).

Main Features

1. Support custom dataset with custom data loader for `multimodal` module
3. Added model failure support to evaluation `output_context.py`
4. Added median relative error diff to evaluation `evaluate_results.py`
5. Added detailed hardware monitoring support with 5s intervals
6. Support CPU base image when using non-GPU machines
7. Updated base images to reduce image size
8. Bug fix for running benchmark on CPU machine.

This release contains 11 [commits](https://github.com/autogluon/autogluon-bench/compare/v0.2.1...v0.2.2) from 3 contributor.

Full Contributor List (ordered by of commits):
Innixma suzhoum yinweisu

This release supports Python versions 3.9. and 3.10.

We hope you enjoy the new features in this release! Please don't hesitate to reach out to us with any questions or feedback.

We're excited to announce the release of AutoGluon-Bench 0.2.1!

For more details, please check out the [README](https://github.com/autogluon/autogluon-bench/blob/master/README.md).

Main Features

1. Supports AMLB `user_dir` by providing `amlb_user_dir` in tabular configs.
2. Supports custom AMLB branch installation.
3. Supports AMLB task fold parallelization runs in `aws` mode, i.e. each instance runs only 1 fold of task
4. Pre-installed framework in docker container for time saving in large scale benchmark.
5. Removed Python3.8 support to align with AMLB.

This release contains 2 [commits](https://github.com/autogluon/autogluon-bench/compare/v0.2.0...v0.2.1) from 1 contributor.

Full Contributor List (ordered by of commits):
suzhoum

This release supports Python versions 3.9. and 3.10.

We hope you enjoy the new features in this release! Please don't hesitate to reach out to us with any questions or feedback.

We're excited to announce the release of AutoGluon-Bench 0.2.0!

For more details, please check out the [README](https://github.com/autogluon/autogluon-bench/blob/master/README.md).

Main Features

1. Push `GPUUtilization` to AWS CloudWatch
2. Added support for python3.10
3. Fix broken python3.8 support
4. Support unit test runs under different python environments
5. Migrate from `setup.py` to `pyproject.toml` with automatic versioning and clearer structure

This release contains 2 [commits](https://github.com/autogluon/autogluon-bench/compare/v0.1.0...v0.2.0) from 1 contributor.

Full Contributor List (ordered by of commits):
suzhoum

This release supports Python versions 3.8, 3.9. and 3.10.

We hope you enjoy the new features in this release! Please don't hesitate to reach out to us with any questions or feedback.

We're excited to announce the release of AutoGluon-Bench 0.1.0!

For more details, please check out the [README](https://github.com/autogluon/autogluon-bench/blob/master/README.md).

Main Features

1. Added evaluation support for tabular module
2. Restructured multimodal results and supported aggregating the results
3. Use validation data in `MultiModalPredictor` when available
4. Added `--wait` option in `agbench run` to block the process for CI integration
5. CLI to generate cloud config file
6. Added `--dev-branch` in `agbench run` to support easy development testing
7. Supports querying instance information for any instance type
8. Adds `--config-file` option to `agbench --destroy-stack`
9. Bug fixes

This release contains 13 [commits](https://github.com/autogluon/autogluon-bench/compare/v0.0.4...v0.1.0) from 3 contributor.

Full Contributor List (ordered by of commits):
suzhoum gidler yinweisu

This release supports Python versions 3.8, and 3.9.

We hope you enjoy the new features in this release! Please don't hesitate to reach out to us with any questions or feedback.

We're excited to announce the release of AutoGluon-Bench 0.0.4! We have done some bug fixes for AWS deployment, and we have also enabled auto release to PyPI!

For more details, please check out the [README](https://github.com/autogluon/autogluon-bench/blob/master/README.md).

Main Features

1. Better support for configuring benchmark, updated sample configuration file to reflect.
2. Bug fixes for AWS deployment
3. Auto release to PyPI upon creating the GitHub release.

This release contains 2 [commits](https://github.com/autogluon/autogluon-bench/compare/v0.0.3...v0.0.4) from 1 contributor.

Full Contributor List (ordered by of commits):
suzhoum

This release supports Python versions 3.8, and 3.9.

We hope you enjoy the new features in this release! Please don't hesitate to reach out to us with any questions or feedback.

We're excited to announce the release of AutoGluon-Bench 0.0.3! We made some major changes to the API and also made it available on PyPI, so please try it out and reach out to us for any feedback!

For more details, please check out the [README](https://github.com/autogluon/autogluon-bench/blob/master/README.md).

Main Features

1. Added CLI tool for running benchmarks and querying job status.
2. Better grouping for benchmarking artifacts saved locally and on S3.
3. Bug fixes.

This release contains 2 [commits](https://github.com/autogluon/autogluon-bench/compare/v0.0.2...v0.0.3) from 1 contributor.

Full Contributor List (ordered by of commits):
suzhoum

This release supports Python versions 3.8, and 3.9.

We hope you enjoy the new features in this release! Please don't hesitate to reach out to us with any questions or feedback.