Mmdeploy

Highlight

- Provide prebuilt packages since v0.5.0
- Decouple `pytorch2onnx` and `onnx2backends`
- Support text detection models PANet, PSENet and DBNet, with CUDA accelerated postprocessing in SDK
- Support MMRotate

Features

- Add prebuild tools (545, 347)
- Experimental executor support in SDK (497)
- Support ViT on ncnn (477, 403)
- Support LiteHRNet on ncnn (316)
- Support more text detection models PANet, PSENet and DBNet, with CUDA accelerated postprocessing in SDK (446, 526, 534)
- Add C API for SDK (388, 535)
- Support ncnn quantization (476)
- Support [RepPoints](https://github.com/open-mmlab/mmdetection/blob/master/configs/reppoints) on TensorRT (#457)
- Support MMRotate on ONNX Runtime and TensorRT (277, 312, 422, 450, 428, 473)
- Support MMRazor (220, 467)

Improvements

- Remove `spdlog` manual installation but still keep it as an option (423, 544)
Users can turn on the following option to use the external spdlog
shell
cmake .. -DMMDEPLOY_SPDLOG_EXTERNAL=ON

- Add SDK python demos (554)
- Add ONNX passes support (390)
- Decouple `pytorch2onnx` and `onnx2backends` (529, 540)
- Add scripts and configs to test metrics of deployed model with all inference backend (425, 302, 551, 542)
- Support MDCN and DeformConv TensorRT FP16 (503, 468)
- Add interactive build script for Linux and NVIDIA platform (399)
- Optimize global average pooling when exporting ONNX (478)
- Refactor `onnx2ncnn`, add test cases and simplify code (436)
- Remove `expand` operation from mmdet rewrite (371)

Bug fixes

- Update CMake scripts to fix building problems (544, 553)
- Make ONNXRuntime wrapper work both for cpu and cuda execution (438, 532)
- Fix PSPNet-TorchScript conversion error (538)
- Resolve the incompatible issue when upgrading MMPose from v0.25.0 to v0.26.0 (518, 527)
- Fix mismatched device issue when testing Mask R-CNN deployed model (511)
- Remove redundant `resize` in mmseg EncoderDecoder rewrite (480)
- Fix display bugs on headless devices (451)
- Fix MMDet3D `pillarencode` deployment failure (331)
- Make the latest `spdlog` compatible (423)
- Fix CI (462, 447, 440, 426, 441)
- Fix a bug that causes exporting to onnx failed with static shape and batch size > 1 (501)
- Make `--work-dir` default to` $pwd` in `tools/deploy.py` (483)

Documentation

- Fix user document errors, reorganize them, update REAME and rewrite the GET_STARTED chapters (418, 482, 509, 531, 547, 543)
- Rewrite the [get_started](https://github.com/open-mmlab/mmdeploy/blob/master/docs/en/01-how-to-build/jetsons.md) for Jetson platforms (#484, 449, 415, 381)
- Fix APIs rendering failure in readthedocs (443)
- Remove '\' in API docstring (495)
- More tutorials in Chinese are checked in - [Tutorial 05: ONNX Model Editing](https://github.com/open-mmlab/mmdeploy/blob/master/docs/zh_cn/05-tutorial/05_onnx_model_editing.md) and [Tutorial 04: onnx custom op](https://github.com/open-mmlab/mmdeploy/blob/master/docs/zh_cn/05-tutorial/04_onnx_custom_op.md) (#508, 517)

Contributors

sanjaypavo PeterH0323 tehkillerbee zytx121 triple-Mu zhiqwang gyf304 lakshanthad Dchaoqun zhouzaida NagatoYuki0943 VVsssssk irexyc RunningLeon hanrui1sensetime lzhangzz grimoire tpoisonooo AllentDan SingleZombie

Improvements

- Add IPython notebook tutorial (234)
- Support detecting TensorRT from `CUDA_TOOLKIT_ROOT_DIR` (357)
- Build onnxruntime backend in GPU dockerfile (366)
- Add CircleCI workflow for linting (348)
- Support saving results when testing the deployed model of MMEdit (336)
- Support GPU postprocessing for instance segmentation (276)

Bug fixes

- Make empty bounding box list allowed in text recognizer and pose detector C API (310, 396)
- Fix the logic of extracting model name from config (394)
- Fix feature test for std::source_location (416)
- Add missing codegen for `sm_53` to support Jetson Nano (407)
- Fix crash caused by accessing the wrong tensor in segmentor C API (363)
- Fix reading mat type from the wrong image in a batch (362)
- Fix missing binary flag when saving temp OpenVINO model (353)
- Fix Windows build for pose demo (307)

Documents

- Refine documents by fixing typos, correcting build commands, and removing redundant doc tree (352, 360, 378, 398)
- Add a tutorial about torch2onnx in Chinese (365)

Contributors
irexyc VVsssssk AllentDan lzhangzz PeterH0323 RunningLeon zly19540609 triple-Mu grimoire hanrui1sensetime SingleZombie Adenialzz tpoisonooo lvhan028 xizi

Features

- Support MMPose model inference in SDK: `HRNet`, `LiteHRNet` and `MSPN`
- Support MMDetection3D: `PointPillars` and `CenterPoint(pillar)`
- Support Andoid platform so as to benefit the development of android apps
- Support `fcn_unet` deployment with dynamic shape
- Support TorchScript


Improvements

- Optimize `TRTMultiLevelRoiAlign` plugin
- Remove `RoiAlign` plugin for ONNXRuntime
- Add `DCN` TensorRT plugin
- Update pad logic in detection heads
- Refactor the rewriter module of Model Converter
- Suppress CMAKE_CUDA_ARCHITECTURES warnings
- Update cmake scripts to ensure that the thirdparty packages are relocatable

Bug fixes

- Fix the crash on the headless installation
- Correct the deployment configs for MMSegmentation
- Optimize prepocess module and fix the potential use-after-free issue
- Resolve the compatibility with torch 1.11
- Fix the errors when deploying yolox model
- Fix the errors occurred during docker build


Documents

- Reorganize the build documents. Add more details about how to build MMDeploy on Linx, Windows and Android platforms
- Publish two chapters about the knowledge of model deployment
- Update the supported model list, including MMSegmentation，MMPose and MMDetection3D
- Translate the tutorial of "How to support new backends" into Chinese
- Update the FAQ

Contributors
irexyc lvhan028 RunningLeon hanrui1sensetime AllentDan grimoire lzhangzz SemyonBevzuk VVsssssk SingleZombie raykindle yydc-0 haofanwang LJoson PeterH0323

Features
- Support for windows platform.(106)
- Support mmpose codebase.(94)
- Support GFL model from mmdetection.(124)
- Support export hardsigmoid in torch<=1.8.(169)


Improvements
- Support mmocr v0.4+.(115)
- Upgrade isort in pre-commit config.(141)
- Opimize delta2bboxes.(152)


Bug fixes
- Fix onnxruntime wrapper for gpu inference. (123)
- Fix ci.(144)
- Fix tests for OpenVINO with python 3.6. (125)
- Added TensorRT version check. (133)
- Fix a type error when computing scale_factor in rewriting interpolate.(185)


Documents
- Add Chinese documents `How_to_support_new_model.md` and `How_to_write_config.md`
(147,137)


Contributors


A total of 19 developers contributed to this release.


grimoire RunningLeon AllentDan lvhan028 hhaAndroid SingleZombie lzhangzz hanrui1sensetime Vvsssssk SemyonBevzuk ypwhs TheSeriousProgrammer matrixgame2018 tehkillerbee uniyushu haofanwang ypwhs zhouzaida q3394101

Features
- Support Nvidia Jetson deployment. (Nano, TX2, Xavier)
- Add Python interface for SDK inference. (27)
- Support yolox on ncnn. (29)
- Support segmentation model UNet. (77)
- Add docker files. (67)

Improvements
- Add coverage report, CI to GitHub repository. (16, 34, 35)
- Refactor the config utilities. (12, 36)
- Remove redundant copy operation when converting model. (61)
- Simplify single batch NMS. (99)

Documents
- Now our English and Chinese documents are available on readthedocs: [English](https://mmdeploy.readthedocs.io/en/latest/) [简体中文](https://mmdeploy.readthedocs.io/zh_CN/latest/)
- Benchmark and tutorial for Nvidia Jetson Nano. (71)
- Fix docstring, links in documents. (18, 32, 60, 84)
- More documents for TensorRT and OpenVINO. (96, 102)

Bug fixes
- Avoid outputting empty tensor in NMS for ONNX Runtime. (42)
- Fix TensorRT 7 SSD. (49)
- Fix mmseg dynamic shape. (57)
- Fix bugs about pplnn. (40, 74)

Contributors

A total of 14 developers contributed to this release.

grimoire RunningLeon AllentDan SemyonBevzuk lvhan028 hhaAndroid Stephenfang51 SingleZombie lzhangzz hanrui1sensetime VVsssssk zhiqwang tehkillerbee Echo-minn

Major Features

- **Fully support OpenMMLab models**

We provide a unified model deployment toolbox for the codebases in OpenMMLab. The supported codebases are listed as below, and more will be added in the future
- [x] MMClassification (== 0.19.0)
- [x] MMDetection (== 2.19.0)
- [x] MMSegmentation (== 0.19.0)
- [x] MMEditing (== 0.11.0)
- [x] MMOCR (== 0.3.0)

- **Multiple inference backends are available**

Models can be exported and run in different backends. The following ones are supported, and more will be taken into consideration
- [x] ONNX Runtime (>= 1.8.0)
- [x] TensorRT (>= 7.2)
- [x] PPLNN (== 0.3.0)
- [x] ncnn (== 20211208)
- [x] OpenVINO (2021 4 LTS)

- **Efficient and highly scalable SDK Framework by C/C++**

All kinds of modules in SDK can be extensible, such as `Transform` for image processing, `Net` for Neural Network inference, `Module` for postprocessing and so on.

Contributors

A total of 11 developers contributed to this release.

grimoire lvhan028 AllentDan VVsssssk SemyonBevzuk lzhangzz RunningLeon SingleZombie del-zhenwu zhouzaida hanrui1sensetime