Optimum

We’re excited to announce the release of **Optimum v1.24.0**. This update expands ONNX-based model capabilities and includes several improvements, bug fixes, and new contributions from the community.

:rocket: New Features & Enhancements
- `ORTQuantizer` now supports models with ONNX subfolders.
- **ONNX Runtime IO Binding support** for all supported Transformers models (no models left behind).
- **SD3 and Flux model support** added to `ORTDiffusionPipeline` enabling latest diffusion-based models.
- **Transformers v4.47 and v4.48 compatibility**, ensuring seamless integration with the latest advancements in Hugging Face's ecosystem.
- **ONNX export support** extended to various models, including Decision Transformer, ModernBERT, Megatron-BERT, Dinov2, OLMo, and many more (see details).

:wrench: Key Fixes & Optimizations
- **Dropped support for Python 3.8**
- **Bug fixes** in `ModelPatcher`, SDXL refiner export, and device checks for improved reliability.

:busts_in_silhouette: New Contributors
A huge thank you to our first-time contributors:
- gabe-l-hart
- ra9hur
- bndos
- mlynatom
- LoSealL
- sjrl
- guangy10
- LRL-ModelCloud
- pragyandev

Your contributions make Optimum better! :tada:

For a detailed list of all changes, please check out the **[full changelog](https://github.com/huggingface/optimum/compare/v1.23.3...v1.24.0)**.

:rocket: Happy optimizing!

What's Changed

<details>

* Onnx granite by gabe-l-hart in https://github.com/huggingface/optimum/pull/2043
* Drop python 3.8 by echarlaix in https://github.com/huggingface/optimum/pull/2086
* Update Dockerfile base image by echarlaix in https://github.com/huggingface/optimum/pull/2089
* add transformers 4.36 tests by echarlaix in https://github.com/huggingface/optimum/pull/2085
* [`fix`] Allow ORTQuantizer over models with subfolder ONNX files by tomaarsen in https://github.com/huggingface/optimum/pull/2094
* SD3 and Flux support by IlyasMoutawwakil in https://github.com/huggingface/optimum/pull/2073
* Remove datasets as required dependency by echarlaix in https://github.com/huggingface/optimum/pull/2087
* Add ONNX Support for Decision Transformer Model by ra9hur in https://github.com/huggingface/optimum/pull/2038
* Generate guidance for flux by IlyasMoutawwakil in https://github.com/huggingface/optimum/pull/2104
* Unbundle inputs generated by `DummyTimestepInputGenerator` by JingyaHuang in https://github.com/huggingface/optimum/pull/2107
* Pass the revision to SentenceTransformer models by bndos in https://github.com/huggingface/optimum/pull/2105
* Rembert onnx support by mlynatom in https://github.com/huggingface/optimum/pull/2108
* fix bug `ModelPatcher` returns empty outputs by LoSealL in https://github.com/huggingface/optimum/pull/2109
* Fix workflow to mark issues as stale by echarlaix in https://github.com/huggingface/optimum/pull/2110
* Remove doc-build by echarlaix in https://github.com/huggingface/optimum/pull/2111
* Downgrade stale bot to v8 and fix permissions by echarlaix in https://github.com/huggingface/optimum/pull/2112
* Update documentation color from google tpu section by echarlaix in https://github.com/huggingface/optimum/pull/2113
* Fix workflow to mark PRs as stale by echarlaix in https://github.com/huggingface/optimum/pull/2116
* Enable transformers v4.47 support by echarlaix in https://github.com/huggingface/optimum/pull/2119
* Add ONNX export support for MGP-STR by xenova in https://github.com/huggingface/optimum/pull/2099
* Add ONNX export support for OLMo and OLMo2 by xenova in https://github.com/huggingface/optimum/pull/2121
* Pass on `model_kwargs` when exporting a SentenceTransformers model by sjrl in https://github.com/huggingface/optimum/pull/2126
* Add ONNX export support for DinoV2, Hiera, Maskformer, PVT, SigLIP, SwinV2, VitMAE, and VitMSN models by xenova in https://github.com/huggingface/optimum/pull/2001
* move check_dummy_inputs_allowed to common export utils by eaidova in https://github.com/huggingface/optimum/pull/2114
* Remove CI macos runners by echarlaix in https://github.com/huggingface/optimum/pull/2129
* Enable GPTQModel by jiqing-feng in https://github.com/huggingface/optimum/pull/2064
* Skip private model loading for external contributors by echarlaix in https://github.com/huggingface/optimum/pull/2130
* fix sdxl refiner export by eaidova in https://github.com/huggingface/optimum/pull/2133
* Export to ExecuTorch: Initial Integration by guangy10 in https://github.com/huggingface/optimum/pull/2090
* Fix AutoModel can't load gptq model due to module prefix mismatch vs AutoModelForCausalLM by LRL-ModelCloud in https://github.com/huggingface/optimum/pull/2146
* Update docker files by echarlaix in https://github.com/huggingface/optimum/pull/2102
* Limit diffusers version by IlyasMoutawwakil in https://github.com/huggingface/optimum/pull/2150
* Add ONNX export support for ModernBERT by xenova in https://github.com/huggingface/optimum/pull/2131
* Allow GPTQModel to auto select Marlin or faster kernels for inference only ops by LRL-ModelCloud in https://github.com/huggingface/optimum/pull/2138
* fix device check by jiqing-feng in https://github.com/huggingface/optimum/pull/2136
* Replace check_if_xxx_greater with is_xxx_version by echarlaix in https://github.com/huggingface/optimum/pull/2152
* Add tf available and version by echarlaix in https://github.com/huggingface/optimum/pull/2154
* Add ONNX export support for `PatchTST` by xenova in https://github.com/huggingface/optimum/pull/2101
* fix infer task from model_name if model from sentence transformer by eaidova in https://github.com/huggingface/optimum/pull/2151
* Unpin diffusers and pass onnx exporters tests by IlyasMoutawwakil in https://github.com/huggingface/optimum/pull/2153
* Uncomment modernbert config by IlyasMoutawwakil in https://github.com/huggingface/optimum/pull/2155
* Skip optimum-benchmark when loading namespace modules by IlyasMoutawwakil in https://github.com/huggingface/optimum/pull/2159
* Fix PR doc upload by regisss in https://github.com/huggingface/optimum/pull/2161
* Move executorch to optimum-executorch by echarlaix in https://github.com/huggingface/optimum/pull/2165
* Adding Onnx Support For Megatron-Bert by pragyandev in https://github.com/huggingface/optimum/pull/2169
* Transformers 4.48 by IlyasMoutawwakil in https://github.com/huggingface/optimum/pull/2158
* Update ort CIs (slow, gpu, train) by IlyasMoutawwakil in https://github.com/huggingface/optimum/pull/2024

</details>

* Add sentence-transformers and timm documentation example by echarlaix in https://github.com/huggingface/optimum/pull/2072
* Create token type ids when not provided by echarlaix in https://github.com/huggingface/optimum/pull/2081
* Add transformers v4.46 support by echarlaix in https://github.com/huggingface/optimum/pull/2078

- Fix compatibility with diffusers < 0.25.0 2063 echarlaix
- Update the habana extra 2077 regisss

**Full Changelog**: https://github.com/huggingface/optimum/compare/v1.23.1...v1.23.2

* Fix doc build by regisss in https://github.com/huggingface/optimum/pull/2050
* Don't hardcode the logger level to INFO let users set TRANSFORMERS_VERBOSITY by tomaarsen in https://github.com/huggingface/optimum/pull/2047
* Add workflow to mark issues as stale by regisss in https://github.com/huggingface/optimum/pull/2051
* Fix onnx export when transformers >= v4.45 (impacting sentence-transformers and timm models) by echarlaix in https://github.com/huggingface/optimum/pull/2053 and https://github.com/huggingface/optimum/pull/2054

ONNX Runtime Diffusion pipeline

Adding `ORTDiffusionPipeline` to simplify diffusers model loading by IlyasMoutawwakil in https://github.com/huggingface/optimum/pull/1960 and https://github.com/huggingface/optimum/pull/2021

diff
model_id = "runwayml/stable-diffusion-v1-5"
- pipeline = ORTStableDiffusionPipeline.from_pretrained(model_id, revision="onnx")
+ pipeline = ORTDiffusionPipeline.from_pretrained(model_id, revision="onnx")
image = pipeline("sailing ship in storm by Leonardo da Vinci").images[0]

What's Changed
* Fix sentence transformers modeling patching for export by echarlaix in https://github.com/huggingface/optimum/pull/1936
* Update optimum intel extra by echarlaix in https://github.com/huggingface/optimum/pull/1935
* Update Habana extra by regisss in https://github.com/huggingface/optimum/pull/1937
* Remove inplace op in mistral patcher by IlyasMoutawwakil in https://github.com/huggingface/optimum/pull/1938
* Fix forward bug in ORTModelForFeatureExtraction by moria97 in https://github.com/huggingface/optimum/pull/1941
* Deprecate ORTModel class by IlyasMoutawwakil in https://github.com/huggingface/optimum/pull/1939
* Remove warning by echarlaix in https://github.com/huggingface/optimum/pull/1945
* Clip vision model onnx export by fxmarty in https://github.com/huggingface/optimum/pull/1920
* Add export test for swin with shifted windows by echarlaix in https://github.com/huggingface/optimum/pull/1942
* Refactor diffusers tasks by IlyasMoutawwakil in https://github.com/huggingface/optimum/pull/1947
* Fix optimizer's command line reading by idruker-cerence in https://github.com/huggingface/optimum/pull/1961
* Fix unmask_unattended_patched signature by fxmarty in https://github.com/huggingface/optimum/pull/1963
* Fix undefined variable in library name inference by IlyasMoutawwakil in https://github.com/huggingface/optimum/pull/1964
* Fix gpt bigcode ONNX export for transformers<4.39.0 by echarlaix in https://github.com/huggingface/optimum/pull/1973
* Support transformers 4.43 by IlyasMoutawwakil in https://github.com/huggingface/optimum/pull/1971
* chore(ci): migrate runner configuration in GitHub workflows by XciD in https://github.com/huggingface/optimum/pull/1978
* Fix typos in quantization.mdx by aldakata in https://github.com/huggingface/optimum/pull/1989
* Update Habana extra in setup.py by regisss in https://github.com/huggingface/optimum/pull/1991
* Follow up the diffusers task refactoring by JingyaHuang in https://github.com/huggingface/optimum/pull/1999
* Transformers 4.44 support by IlyasMoutawwakil in https://github.com/huggingface/optimum/pull/1996
* Modify token classification processor default dataset args by echarlaix in https://github.com/huggingface/optimum/pull/2005
* Fix TFLite tests by IlyasMoutawwakil in https://github.com/huggingface/optimum/pull/2007
* Fix attribute name from `inputs_names` to `input_names` by J4BEZ in https://github.com/huggingface/optimum/pull/2010
* Fix typo in BetterTransformer's overview docs by ftnext in https://github.com/huggingface/optimum/pull/2015
* Apply deprecated `evaluation_strategy` by muellerzr in https://github.com/huggingface/optimum/pull/1819
* Update transformers imports for `deepspeed` and `is_torch_xla_available` by Rohan138 in https://github.com/huggingface/optimum/pull/2012
* Add quanto install and instructions by dacorvo in https://github.com/huggingface/optimum/pull/1976

New Contributors
* moria97 made their first contribution in https://github.com/huggingface/optimum/pull/1941
* XciD made their first contribution in https://github.com/huggingface/optimum/pull/1978
* zhenglongjiepheonix made their first contribution in https://github.com/huggingface/optimum/pull/1933
* aldakata made their first contribution in https://github.com/huggingface/optimum/pull/1989
* J4BEZ made their first contribution in https://github.com/huggingface/optimum/pull/2010
* ftnext made their first contribution in https://github.com/huggingface/optimum/pull/2015
* muellerzr made their first contribution in https://github.com/huggingface/optimum/pull/1819
* Rohan138 made their first contribution in https://github.com/huggingface/optimum/pull/2012

**Full Changelog**: https://github.com/huggingface/optimum/compare/v1.21.4...v1.22.0