Optimum-benchmark

What's Changed
* Fix images building by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/242
* Faster quality check by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/243
* Decode output of `nvmlDeviceGetName` to avoid JSON serialize issue by KeitaW in https://github.com/huggingface/optimum-benchmark/pull/240
* Fix makefile typo by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/244
* fix neural compressor backend by baptistecolle in https://github.com/huggingface/optimum-benchmark/pull/245
* Update cuda images by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/246
* Add t4 for llm perf leaderboard by baptistecolle in https://github.com/huggingface/optimum-benchmark/pull/238
* add optimum-intel ipex backend into benchmark by yao-matrix in https://github.com/huggingface/optimum-benchmark/pull/250
* WIP fix rocm runners by baptistecolle in https://github.com/huggingface/optimum-benchmark/pull/249
* Code Style by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/254
* Update ROCm by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/253
* Build from source quantization packages by baptistecolle in https://github.com/huggingface/optimum-benchmark/pull/239
* Fix py-txi ci by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/255
* Fix API tests on ROCm by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/256
* 1. refine cpu Dockerfile for better performance 2. add ipex_bert example by yao-matrix in https://github.com/huggingface/optimum-benchmark/pull/257
* Add support for intel in leaderboard by baptistecolle in https://github.com/huggingface/optimum-benchmark/pull/248
* fix broken canonical list by baptistecolle in https://github.com/huggingface/optimum-benchmark/pull/262
* Fix broken canonical list by baptistecolle in https://github.com/huggingface/optimum-benchmark/pull/264
* Fix issue with CodeCarbon lock by regisss in https://github.com/huggingface/optimum-benchmark/pull/265
* Set is_distributed false by default in vllm by asesorov in https://github.com/huggingface/optimum-benchmark/pull/266
* fix broken cuda and rocm images by baptistecolle in https://github.com/huggingface/optimum-benchmark/pull/263
* Styling by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/267
* Labeling system in CI by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/268
* Multi-gpu vllm by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/269
* fix multi gpu ipc by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/270
* Allow multiple runs and handle connection communication errors by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/271
* Removing barriers by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/273
* Update readme with IPEX by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/274
* Distributed trt-llm by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/275
* ipex backend enhancements by yao-matrix in https://github.com/huggingface/optimum-benchmark/pull/272
* Bump version by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/278
* Pass backend name to EnergyTracker in Training scenario by asesorov in https://github.com/huggingface/optimum-benchmark/pull/279
* move to new runners by glegendre01 in https://github.com/huggingface/optimum-benchmark/pull/281
* Markdown Report by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/280
* dev version by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/284
* Using intermediate env vars in CI by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/290
* remove old code linked to llm-perf leaderboard by baptistecolle in https://github.com/huggingface/optimum-benchmark/pull/291
* Image Text To Text Support by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/296
* Feat: reimplement vllm backend beam search using logprobs by vicoooo26 in https://github.com/huggingface/optimum-benchmark/pull/293
* Add the logic for Energy Star by regisss in https://github.com/huggingface/optimum-benchmark/pull/261
* Add torchao to optimum as a pytorch backend configuration by jerryzh168 in https://github.com/huggingface/optimum-benchmark/pull/297
* fix llamacpp and windows libuv by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/298
* Remove DP vs TP distinction and simplify aggregation across processes by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/299
* Fix misc test by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/300
* Adding latency and memory to energy star by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/302
* Add per_step diffusion measurments by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/303
* Fixes by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/304
* Secure Instinct CI by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/301
* Remove non maintained backends (llm-swarm, inc) by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/305
* Test examples by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/306
* Optional backend kwargs by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/307
* Fix trt llm by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/308
* Protect hf token by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/309
* Preparing for version 0.5 and checking CI by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/310

New Contributors
* KeitaW made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/240
* yao-matrix made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/250
* asesorov made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/266
* glegendre01 made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/281
* vicoooo26 made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/293
* jerryzh168 made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/297

**Full Changelog**: https://github.com/huggingface/optimum-benchmark/compare/v0.4.0...v0.5.0

What's Changed
* Refactor backends and add `load` tracking by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/227
* Update readme by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/228
* Update vllm backend to support offline and online serving modes by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/232
* Misc CI updates and multi-platform support by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/233
* Add llama.cpp backend by baptistecolle in https://github.com/huggingface/optimum-benchmark/pull/231
* Misc changes and fixes for llama cpp by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/236
* release by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/237

New Contributors
* baptistecolle made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/231

**Full Changelog**: https://github.com/huggingface/optimum-benchmark/compare/v0.3.1...v0.4.0

What's Changed
* Fix per token latency by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/223
* Per token latency outliers by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/225
* Patch release by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/224

**Full Changelog**: https://github.com/huggingface/optimum-benchmark/compare/v0.3.0...v0.3.1

What's Changed
* Remove experiment schema by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/210
* Numactl support by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/211
* Fix sentence transformers models by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/212
* Enable security checks by mfuntowicz in https://github.com/huggingface/optimum-benchmark/pull/216
* Fix `PyTorchBackend` TP vs DP inputs distribution across replicas and shards by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/218
* Pin eager attn in torch-ort backend by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/219
* Fix INC by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/220
* bump version 0.3.0 by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/221

New Contributors
* mfuntowicz made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/216

**Full Changelog**: https://github.com/huggingface/optimum-benchmark/compare/v0.2.1...v0.3.0

What's Changed
* Llm perf update by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/206
* Fix diffusers repo id naming by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/208
* Release by IlyasMoutawwakil in https://github.com/huggingface/optimum-benchmark/pull/209


**Full Changelog**: https://github.com/huggingface/optimum-benchmark/compare/v0.2.0...v0.2.1

New Contributors
* karthickai made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/149
* aliabdelkader made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/144
* lopozz made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/141
* helena-intel made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/172
* regisss made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/169
* yamaura made their first contribution in https://github.com/huggingface/optimum-benchmark/pull/192

**Full Changelog**: https://github.com/huggingface/optimum-benchmark/compare/0.0.1...v0.2.0