Intel® Low Precision Optimization Tool (iLiT) is an open-sourced python library which is intended to deliver a unified low-precision inference solution cross multiple Intel optimized DL frameworks on both CPU and GPU. It supports automatic accuracy-driven tuning strategies, along with additional objectives like performance, model size, or memory footprint. It also provides the easy extension capability for new backends, tuning strategies, metrics and objectives.
Feature List:
* Unified low precision quantization interface cross multiple Intel optimized frameworks (TensorFlow, PyTorch, and MXNet)
* Built-in tuning strategies, including Basic, Bayesian, and MSE
* Built-in evaluation metrics, including TopK (image classification), F1 (NLP), and CocoMAP (object detection)
* Built-in tuning objectives, including Performance, ModelSize, and Footprint
* Extensible API design to add new strategy, framework backend, metric, and objective
* KL-divergence calibration for TensorFlow and MXNet
* Tuning process resume from certain checkpoint
Supported Models:
Model | Framework | Model | Framework | Model | Framework
-- | -- | -- | -- | -- | --
ResNet50 V1 | MXNet | BERT-Large RTE | PyTorch | ResNet18 | PyTorch
MobileNet V1 | MXNet | BERT-Large QNLI | PyTorch | ResNet50 V1 | TensorFlow
MobileNet V2 | MXNet | BERT-Large CoLA | PyTorch | ResNet50 V1.5 | TensorFlow
SSD-ResNet50 | MXNet | BERT-Base SST-2 | PyTorch | ResNet101 | TensorFlow
SqueezeNet V1 | MXNet | BERT-Base RTE | PyTorch | Inception V1 | TensorFlow
ResNet18 | MXNet | BERT-Base STS-B | PyTorch | Inception V2 | TensorFlow
Inception V3 | MXNet | BERT-Base CoLA | PyTorch | Inception V3 | TensorFlow
DLRM | PyTorch | BERT-Base MRPC | PyTorch | Inception V4 | TensorFlow
BERT-Large MRPC | PyTorch | ResNet101 | PyTorch | Inception ResNet V2 | TensorFlow
BERT-Large SQUAD | PyTorch | ResNet50 V1.5 | PyTorch | SSD ResNet50 V1 | TensorFlow
Known Issues:
* Statistics collection for KL algorithm is slow in TensorFlow due to lack of tensor inspector APIs
* MSE tuning strategy is not supported in PyTorch
Validated Configurations:
* Python 3.6 & 3.7
* Centos 7
* TensorFlow 1.15, 2.0 and 2.1
* PyTorch 1.5
* MxNet 1.6
Distribution:
| Channel | Links | Install Command
-- | -- | -- | --
Source | Github | https://github.com/intel/lp-opt-tool.git | $ git clone https://github.com/intel/lp-opt-tool.git
Binary | Pip | https://pypi.org/project/ilit | $ pip install ilit
Binary | Conda | https://anaconda.org/intel/ilit | $ conda config --add channels intel $ conda install ilit
Contact:
Please feel free to contact ilit.maintainersintel.com, if you get any questions.