Optimum-benchmark

What's Changed
* Refactor backends and add `load` tracking by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/227
* Update readme by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/228
* Update vllm backend to support offline and online serving modes by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/232
* Misc CI updates and multi-platform support by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/233
* Add llama.cpp backend by baptistecolle in https://github.com/huggingface/optimum-benchmark/pull/231
* Misc changes and fixes for llama cpp by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/236
* release by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/237

New Contributors
* baptistecolle made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/231

**Full Changelog**: https://github.com/huggingface/optimum-benchmark/compare/v0.3.1...v0.4.0

What's Changed
* Fix per token latency by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/223
* Per token latency outliers by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/225
* Patch release by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/224

**Full Changelog**: https://github.com/huggingface/optimum-benchmark/compare/v0.3.0...v0.3.1

What's Changed
* Remove experiment schema by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/210
* Numactl support by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/211
* Fix sentence transformers models by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/212
* Enable security checks by mfuntowicz in https://github.com/huggingface/optimum-benchmark/pull/216
* Fix `PyTorchBackend` TP vs DP inputs distribution across replicas and shards by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/218
* Pin eager attn in torch-ort backend by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/219
* Fix INC by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/220
* bump version 0.3.0 by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/221

New Contributors
* mfuntowicz made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/216

**Full Changelog**: https://github.com/huggingface/optimum-benchmark/compare/v0.2.1...v0.3.0

What's Changed
* Llm perf update by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/206
* Fix diffusers repo id naming by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/208
* Release by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/209


**Full Changelog**: https://github.com/huggingface/optimum-benchmark/compare/v0.2.0...v0.2.1

New Contributors
* karthickai made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/149
* aliabdelkader made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/144
* lopozz made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/141
* helena-intel made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/172
* regisss made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/169
* yamaura made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/192

**Full Changelog**: https://github.com/huggingface/optimum-benchmark/compare/0.0.1...v0.2.0