ONNX export from `nn.Module`
A function is exposed to programmatically export any `nn.Module` (e.g. models coming from Transformers, but modified). This is useful in case you need to do some modifications on models loaded from the Hub before exporting. Example:
python
from transformers import AutoModelForImageClassification
from optimum.exporters.onnx import onnx_export_from_model
model = AutoModelForImageClassification.from_pretrained("google/vit-base-patch16-224")
Here one could do any modification on the model before the export.
onnx_export_from_model(model, output="vit_onnx")
* Enable model ONNX export by echarlaix in https://github.com/huggingface/optimum/pull/1649
ONNX export with static shapes
The Optimum ONNX export CLI allows to disable dynamic shape for inputs/outputs:
optimum-cli export onnx --model timm/ese_vovnet39b.ra_in1k out_vov --no-dynamic-axes
This is useful if the exported model is to be consumed by a runtime that does not support dynamic shapes. The static shape can be specified e.g. with `--batch_size 1` . See all the shape options in `optimum-cli export onnx --help`.
* Enable export of model with fixed shape by mht-sharma in https://github.com/huggingface/optimum/pull/1643
BF16 ONNX export
The Optimum ONNX export now supports BF16 export on CPU and GPU. Beware though that ONNX Runtime is most often not able to consume the models as some operation are not implemented in this data type, although the exported models comply with ONNX standard. This is useful if you are developing a runtime that consomes BF16 ONNX models.
Example:
optimum-cli export onnx --model bert-base-uncased --dtype bf16 bert_onnx
* BF16 support in the ONNX export by fxmarty in https://github.com/huggingface/optimum/pull/1654
ONNX export for news models
You can now [export to ONNX](https://huggingface.co/docs/optimum/main/en/exporters/onnx/overview) table-transformer, bart for text-classification.
* Add ONNX export for table-transformer by xenova in https://github.com/huggingface/optimum/pull/1616
* Reactivate BART Onnx Export by claeyzre in https://github.com/huggingface/optimum/pull/1666
Sentence Transformers ONNX export
* Fix sentence transformers ONNX export by fxmarty in https://github.com/huggingface/optimum/pull/1632
* Bump sentence-transformers ONNX opset by fxmarty in https://github.com/huggingface/optimum/pull/1634
* Pass `trust_remote_code` to sentence transformers export by xenova in https://github.com/huggingface/optimum/pull/1677
* Fix library detection by fxmarty in https://github.com/huggingface/optimum/pull/1690
Timm models support with ONNX Runtime
[Timm](https://github.com/huggingface/pytorch-image-models) models can now be run through ONNX Runtime with the class `ORTModelForImageClassification`:
python
from urllib.request import urlopen
import timm
import torch
from PIL import Image
from optimum.onnxruntime import ORTModelForImageClassification
Export the model to ONNX under the hood with export=True.
model = ORTModelForImageClassification.from_pretrained("timm/resnext101_64x4d.c1_in1k", export=True)
Get model specific transforms (normalization, resize).
data_config = timm.data.resolve_data_config(pretrained_cfg=model.config.pretrained_cfg)
transforms = timm.data.create_transform(**data_config, is_training=False)
img = Image.open(
urlopen("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png")
)
output = model(transforms(img).unsqueeze(0)).logits
top5_probabilities, top5_class_indices = torch.topk(torch.softmax(output, dim=1) * 100, k=5)
* Add Timm support in ORTModelForImageClassification by mht-sharma in https://github.com/huggingface/optimum/pull/1578
Other changes and bugfixes
* Modify SEW-D model for tests by echarlaix in https://github.com/huggingface/optimum/pull/1601
* Add phi and mixtral model type to normalizedconfig by changwangss in https://github.com/huggingface/optimum/pull/1625
* Remove "to ONNX" from info message when exporting model by helena-intel in https://github.com/huggingface/optimum/pull/1627
* Modify model id for test by echarlaix in https://github.com/huggingface/optimum/pull/1628
* Fix cupy detection by fxmarty in https://github.com/huggingface/optimum/pull/1635
* Fix ORT detection by fxmarty in https://github.com/huggingface/optimum/pull/1636
* Enable sdpa export for SD unet component by echarlaix in https://github.com/huggingface/optimum/pull/1637
* [ORT] Improve dummy mask & add tips for attention fusion in the doc by JingyaHuang in https://github.com/huggingface/optimum/pull/1640
* Improve error message by Almonok in https://github.com/huggingface/optimum/pull/1623
* Add `input_labels` input to SAM model export by xenova in https://github.com/huggingface/optimum/pull/1638
* Fix c4 dataset loading by SunMarc in https://github.com/huggingface/optimum/pull/1646
* Avoid loading onnx file in weight deduplication if not necessary by fxmarty in https://github.com/huggingface/optimum/pull/1648
* Allow lower ONNX opsets by fxmarty in https://github.com/huggingface/optimum/pull/1650
* Remove abstract decorator from `_export` by JingyaHuang in https://github.com/huggingface/optimum/pull/1652
* Add rjieba install by mht-sharma in https://github.com/huggingface/optimum/pull/1661
* Fix wikitext2 processing by SunMarc in https://github.com/huggingface/optimum/pull/1663
* Fix: local variable 'dataset' referenced before assignment by hiyouga in https://github.com/huggingface/optimum/pull/1600
* Support float16 images in StableDiffusionXLWatermarker by jambayk in https://github.com/huggingface/optimum/pull/1603
* Extend autocast check to cover more platforms like XPU by hoshibara in https://github.com/huggingface/optimum/pull/1639
* Support IO Binding for ORTModelForCTC by vidalmaxime in https://github.com/huggingface/optimum/pull/1629
* Add fp16 support for split cache by PatriceVignola in https://github.com/huggingface/optimum/pull/1602
* ORTModelForFeatureExtraction always exports as transformers models by fxmarty in https://github.com/huggingface/optimum/pull/1684
* Avoid overriding model_type in TasksManager by fxmarty in https://github.com/huggingface/optimum/pull/1647
* Fix gptq device_map = "cpu" by SunMarc in https://github.com/huggingface/optimum/pull/1662
* CI: Avoid iterating over a mutated iterable by fxmarty in https://github.com/huggingface/optimum/pull/1683
* Add option to disable ONNX constant folding by fxmarty in https://github.com/huggingface/optimum/pull/1682
* re-enable decoder sequence classification by dwyatte in https://github.com/huggingface/optimum/pull/1679
* Move & rename `onnx_export` by fxmarty in https://github.com/huggingface/optimum/pull/1685
* Update standardize_model_attributes by mht-sharma in https://github.com/huggingface/optimum/pull/1686
* Fix: AttributeError: module 'packaging' has no attribute 'version' by soulteary in https://github.com/huggingface/optimum/pull/1660
* Disable failing test & free space when building documentation by fxmarty in https://github.com/huggingface/optimum/pull/1693
* Fix no space left on device in actions by fxmarty in https://github.com/huggingface/optimum/pull/1694
* Add end-to-end Marlin benchmark by fxmarty in https://github.com/huggingface/optimum/pull/1695
* Fix main doc build by fxmarty in https://github.com/huggingface/optimum/pull/1697
* Update optimum-intel requirements by echarlaix in https://github.com/huggingface/optimum/pull/1699
New Contributors
* tomaarsen made their first contribution in https://github.com/huggingface/optimum/pull/1597
* helena-intel made their first contribution in https://github.com/huggingface/optimum/pull/1627
* Almonok made their first contribution in https://github.com/huggingface/optimum/pull/1623
* hiyouga made their first contribution in https://github.com/huggingface/optimum/pull/1600
* jambayk made their first contribution in https://github.com/huggingface/optimum/pull/1603
* hoshibara made their first contribution in https://github.com/huggingface/optimum/pull/1639
* vidalmaxime made their first contribution in https://github.com/huggingface/optimum/pull/1629
* PatriceVignola made their first contribution in https://github.com/huggingface/optimum/pull/1602
* claeyzre made their first contribution in https://github.com/huggingface/optimum/pull/1666
* dwyatte made their first contribution in https://github.com/huggingface/optimum/pull/1679
* soulteary made their first contribution in https://github.com/huggingface/optimum/pull/1660
**Full Changelog**: https://github.com/huggingface/optimum/compare/v1.16.0...v1.17.0