What's Changed
This update improves TKML's support for LLMs and significantly improves data quality for mass-benchmarking scenarios.
LLM Support:
* Add LLaMA2 models by jeremyfowers in https://github.com/onnx/turnkeyml/pull/130
* Add phi2 to transformers corpus by jeremyfowers in https://github.com/onnx/turnkeyml/pull/135
* Fix LLM support for ONNX benchmarking by jeremyfowers in https://github.com/onnx/turnkeyml/pull/134
Data Quality Improvements:
* Add cache warmup for OnnxRuntime Benchmarking by ramkrishna2910 in https://github.com/onnx/turnkeyml/pull/133
* Save the exception into error_log when the subprocess is killed by jeremyfowers in https://github.com/onnx/turnkeyml/pull/139
* Save device name stat prior to running stages by jeremyfowers in https://github.com/onnx/turnkeyml/pull/138
* Capture PyTorch benchmarking outputs into a log file by jeremyfowers in https://github.com/onnx/turnkeyml/pull/121
Quality of Life
* Set a default benchmarking timeout by danielholanda in https://github.com/onnx/turnkeyml/pull/124
Bug fixes
* Ensure that `turnkey cache benchmark` can also target individual builds by danielholanda in https://github.com/onnx/turnkeyml/pull/123
* Update status.py to safely access eval stats for status printout by jeremyfowers in https://github.com/onnx/turnkeyml/pull/122
* Avoid parameter count overflow on `onnx` models by danielholanda in https://github.com/onnx/turnkeyml/pull/126
* Fix hang on OEM info read by ramkrishna2910 in https://github.com/onnx/turnkeyml/pull/128
Breaking Changes
* Plugin interface: `BaseRT.device_name()` is now a static method instead of a property.
**Full Changelog**: https://github.com/onnx/turnkeyml/compare/v1.1.3...v2.0.0