Highlight
- Support more platforms and devices: `RISC-V`, `Apple M1`, `Huawei Ascend310` and `Rockchip RK3588`
Features
- Support more models on ONNX Runtime and TensorRT
- mmdetection `DETR` (924)
- mmclassification `Swin Transformer` (911)
- mmdetection3d `pointpillars (nus version)` (319)
- Support more platforms and devices:
- `RISC-V` via `ncnn` (910)
- `Apple M1` (760)
- `Huawei Ascend310` (747)
- `Rockchip RK3588` (865)
- Add `TorchScript` SDK inference backend (890)
- Experimental support for fusing transformations in preprocess pipeline by [CVFusion](https://github.com/OpenComputeLab/CVFusion) (#741)
Improvements
- Support multi-label classification in SDK (950)
- Add the following scripts to simplify mmdeploy installation for some scenarios: (919)
<div class="okr-block-clipboard" data-okr="%7B%22okrDelta%22%3A%5B%7B%22lineType%22%3A%22unsupport%22%2C%22lineOptions%22%3A%7B%7D%2C%22lineContent%22%3A%5B%5D%7D%5D%2C%22businessKey%22%3A%22lark-doc%22%7D"></div><div data-zone-id="0" data-line-index="0" style="white-space: pre;">
script | OS version
-- | --
build_ubuntu_x64_ncnn.py | 18.04/20.04
build_ubuntu_x64_ort.py | 18.04/20.04
build_ubuntu_x64_pplnn.py | 18.04/20.04
build_ubuntu_x64_torchscript.py | 18.04/20.04
</div>
- Add scaled dot-product attention operator for TensorRT (949)
- Support model batch inference profiling (868)
shell
profile the latency of resnet18-tensorrt model with batch size 4
python tools/profiler.py \
configs/mmcls/classification_tensorrt_dynamic-224x224-224x224.py \
../mmclassification/configs/resnet/resnet18_8xb32_in1k.py \
{/the/path/of/an/image/directory} \
--model {work-dirs}/mmcls/resnet/trt/end2end.engine \
--device cuda \
--shape 224x224 \
--num-iter 100 \
--warmup 10 \
--batch-size 4
Bug fixes
- Fix CI errors (985, 983, 977, 987, 966, 945)
- Fix missing `sqrt` in `PAAHead` (984)
- Fix `nms_rotated` logic when no bbox is detected (976)
- Fix rewrite for `torch.Tensor.__setitem__` in some corner cases (964, 941)
- Disable ONNX optimizer when converting model to ncnn (961)
- Fix regression test (958)
- Disable cublaslt for CUDA 10.2 (947)
- Stop sorting dataset by default & set `test_mode` for mmdet pipelines (920)
- Resolve the issue (909) - ` ValueError: cpu is invalid for the backend tensorrt.` when exporting SDK meta info (912)
- Validate the device id when the inference backend is TensorRT or OpenVINO (886)
- Fix mmdeploy_pplnn_net build error when target device is CPU (896)
- Replace `adaptive_avg_pool2d` with `avg_pool2d` to support exporting ONNX with dynamic shape (857)
Document
- Clarify arguments in model conversion (956, 940)
- Add tutorial in Chinese about "[How to write a customized TensorRT plugin" ](https://github.com/open-mmlab/mmdeploy/blob/master/docs/zh_cn/tutorial/07_write_a_plugin.md)(#290)
- Keep cmake build option in a separate document [cmake_option](https://github.com/open-mmlab/mmdeploy/blob/master/docs/en/01-how-to-build/cmake_option.md). (#832)
- Add project architecture (882)
- Sync Enginsh and Chinese documents (842)
- Correct build-demo commands in [prebuilt_package_windows.md](https://github.com/open-mmlab/mmdeploy/blob/master/docs/en/02-how-to-run/prebuilt_package_windows.md) (#879)
- Fix the wrong argument in model quantization document (866)
Known issues
- `DETR` deployment failed both via ONNX Runtime and TensorRT (1011, [pytorch 84563](https://github.com/pytorch/pytorch/issues/84563))
Contributors
OldDreamInWind liu-mengyang gy-7 Groexhy munhou miraclezqc VVsssssk hanrui1sensetime tpoisonooo grimoire irexyc RunningLeon AllentDan lzhangzz lvhan028