Mmdet

Highlights

- Support new methods: [SCNet](https://arxiv.org/abs/2012.10150), [Sparse R-CNN](https://arxiv.org/abs/2011.12450)
- Move `train_cfg` and `test_cfg` into model in configs
- Support to visualize results based on prediction quality

New Features

- Support [SCNet](https://arxiv.org/abs/2012.10150) (#4356)
- Support [Sparse R-CNN](https://arxiv.org/abs/2011.12450) (#4219)
- Support evaluate mAP by multiple IoUs (4398)
- Support concatenate dataset for testing (4452)
- Support to visualize results based on prediction quality (4441)
- Add ONNX simplify option to Pytorch2ONNX script (4468)
- Add hook for checking compatibility of class numbers in heads and datasets (4508)

Bug Fixes

- Fix CPU inference bug of Cascade RPN (4410)
- Fix NMS error of CornerNet when there is no prediction box (4409)
- Fix TypeError in CornerNet inference (4411)
- Fix bug of PAA when training with background images (4391)
- Fix the error that the window data is not destroyed when `out_file is not None` and `show==False` (4442)
- Fix order of NMS `score_factor` that will decrease the performance of YOLOv3 (4473)
- Fix bug in HTC TTA when the number of detection boxes is 0 (4516)
- Fix resize error in mask data structures (4520)

Improvements

- Allow to customize classes in LVIS dataset (4382)
- Add tutorials for building new models with existing datasets (4396)
- Add CPU compatibility information in documentation (4405)
- Add documentation of deprecated `ImageToTensor` for batch inference (4408)
- Add more details in documentation for customizing dataset (4430)
- Switch `imshow_det_bboxes` visualization backend from OpenCV to Matplotlib (4389)
- Deprecate `ImageToTensor` in `image_demo.py` (4400)
- Move train_cfg/test_cfg into model (4347, 4489)
- Update docstring for `reg_decoded_bbox` option in bbox heads (4467)
- Update dataset information in documentation (4525)
- Release pre-trained R50 and R101 PAA detectors with multi-scale 3x training schedules (4495)
- Add guidance for speed benchmark (4537)

New Contributors
* kazet made their first contribution in https://github.com/open-mmlab/mmdetection/pull/4429
* LutingWang made their first contribution in https://github.com/open-mmlab/mmdetection/pull/4474
* digantamisra98 made their first contribution in https://github.com/open-mmlab/mmdetection/pull/4510

**Full Changelog**: https://github.com/open-mmlab/mmdetection/compare/v2.8.0...v2.9.0

Highlights

- Support new methods: [Cascade RPN](https://arxiv.org/abs/1909.06720), [TridentNet](https://arxiv.org/abs/1901.01892)

New Features

- Support [Cascade RPN](https://arxiv.org/abs/1909.06720) (#1900)
- Support [TridentNet](https://arxiv.org/abs/1901.01892) (#3313)

Bug Fixes

- Fix bug of show result in async_benchmark (4367)
- Fix scale factor in MaskTestMixin (4366)
- Fix but when returning indices in `multiclass_nms` (4362)
- Fix bug of empirical attention in resnext backbone error (4300)
- Fix bug of `img_norm_cfg` in FCOS-HRNet models with updated performance and models (4250)
- Fix invalid checkpoint and log in Mask R-CNN models on Cityscapes dataset (4287)
- Fix bug in distributed sampler when dataset is too small (4257)
- Fix bug of 'PAFPN has no attribute extra_convs_on_inputs' (4235)

Improvements

- Update model url from aws to aliyun (4349)
- Update ATSS for PyTorch 1.6+ (4359)
- Update script to install ruby in pre-commit installation (4360)
- Delete deprecated `mmdet.ops` (4325)
- Refactor hungarian assigner for more general usage in Sparse R-CNN (4259)
- Handle scipy import in DETR to reduce package dependencies (4339)
- Update documentation of usages for config options after MMCV (1.2.3) supports overriding list in config (4326)
- Update pre-train models of faster rcnn trained on COCO subsets (4307)
- Avoid zero or too small value for beta in Dynamic R-CNN (4303)
- Add doccumentation for Pytorch2ONNX (4271)
- Add deprecated warning FPN arguments (4264)
- Support returning indices of kept bboxes when using nms (4251)
- Update type and device requirements when creating tensors `GFLHead` (4210)
- Update device requirements when creating tensors in `CrossEntropyLoss` (4224)

New Contributors
* ggjy made their first contribution in https://github.com/open-mmlab/mmdetection/pull/4224
* ace7chan made their first contribution in https://github.com/open-mmlab/mmdetection/pull/4210
* luuuyi made their first contribution in https://github.com/open-mmlab/mmdetection/pull/4235
* AlekseySh made their first contribution in https://github.com/open-mmlab/mmdetection/pull/4265
* filaPro made their first contribution in https://github.com/open-mmlab/mmdetection/pull/4282
* SiriusKY made their first contribution in https://github.com/open-mmlab/mmdetection/pull/4283

**Full Changelog**: https://github.com/open-mmlab/mmdetection/compare/v2.7.0...v2.8.0

Highlights

- Support new method: [DETR](https://arxiv.org/abs/2005.12872), [ResNest](https://arxiv.org/abs/2004.08955), Faster R-CNN DC5.
- Support YOLO, Mask R-CNN, and Cascade R-CNN models exportable to ONNX.

New Features

- Support [DETR](https://arxiv.org/abs/2005.12872) (#4201, 4206)
- Support to link the best checkpoint in training (3773)
- Support to override config through options in inference.py (4175)
- Support YOLO, Mask R-CNN, and Cascade R-CNN models exportable to ONNX (4087, 4083)
- Support [ResNeSt](https://arxiv.org/abs/2004.08955) backbone (#2959)
- Support unclip-border box regression (4076)
- Add TP/FP functions in evaluating AP (4069)
- Support mixed-precision training of SSD detector with other backbones (4081)
- Add Faster R-CNN DC5 models (4043)

Bug Fixes

- Fix bug of `gpu_id` in distributed training mode (4163)
- Support Albumentations with version higher than 0.5 (4032)
- Fix num_classes bug in faster rcnn config (4088)
- Update code in docs/2_new_data_model.md (4041)

Improvements

- Ensure DCN offset to have similar type as features in VFNet (4198)
- Add config links in README files of models (4190)
- Add tutorials for loss conventions (3818)
- Add solution to installation issues in 30-series GPUs (4176)
- Update docker version in get_started.md (4145)
- Add model statistics and polish some titles in configs README (4140)
- Clamp neg probability in FreeAnchor (4082)
- Speed up expanding large images (4089)
- Fix Pytorch 1.7 incompatibility issues (4103)
- Update troubleshooting page to resolve segmentation fault (4055)
- Update aLRP-Loss in the project page (4078)
- Clean duplicated `reduce_mean` function (4056)
- Refactor Q&A (4045)

New Contributors
* miaosakurai made their first contribution in https://github.com/open-mmlab/mmdetection/pull/4041
* wdmwhh made their first contribution in https://github.com/open-mmlab/mmdetection/pull/4054
* cancam made their first contribution in https://github.com/open-mmlab/mmdetection/pull/4078
* Sundrops made their first contribution in https://github.com/open-mmlab/mmdetection/pull/4081
* renjithbaby23 made their first contribution in https://github.com/open-mmlab/mmdetection/pull/4145
* sunnyxiaohu made their first contribution in https://github.com/open-mmlab/mmdetection/pull/4161

**Full Changelog**: https://github.com/open-mmlab/mmdetection/compare/v2.6.0...v2.7.0

Highlights

- Support new method: [VarifocalNet](https://arxiv.org/abs/2008.13367).
- Refactored documentation with more tutorials.

New Features

- Support GIoU calculation in `BboxOverlaps2D`, and re-implement `giou_loss` using `bbox_overlaps` (3936)
- Support random sampling in CPU mode (3948)
- Support VarifocalNet (3666, 4024)

Bug Fixes

- Fix SABL validating bug in Cascade R-CNN (3913)
- Avoid division by zero in PAA head when num_pos=0 (3938)
- Fix temporary directory bug of multi-node testing error (4034, 4017)
- Fix `--show-dir` option in test script (4025)
- Fix GA-RetinaNet r50 model url (3983)
- Update code in docs and fix broken urls (3947)

Improvements

- Refactor pytorch2onnx API into `mmdet.core.export` and use `generate_inputs_and_wrap_model` for pytorch2onnx (3857, 3912)
- Update RPN upgrade scripts for v2.5.0 compatibility (3986)
- Use mmcv `tensor2imgs` (4010)
- Update test robustness (4000)
- Update trouble shooting page (3994)
- Accelerate PAA training speed (3985)
- Support batch_size > 1 in validation (3966)
- Use RoIAlign implemented in MMCV for inference in CPU mode (3930)
- Documentation refactoring (4031)


New Contributors
* wangruohui made their first contribution in https://github.com/open-mmlab/mmdetection/pull/3937
* hubutui made their first contribution in https://github.com/open-mmlab/mmdetection/pull/3947
* hyz-xmaster made their first contribution in https://github.com/open-mmlab/mmdetection/pull/3666
* AemikaChow made their first contribution in https://github.com/open-mmlab/mmdetection/pull/4031

**Full Changelog**: https://github.com/open-mmlab/mmdetection/compare/v2.5.0...v2.6.0

Highlights

- Support new methods: [YOLACT](https://arxiv.org/abs/1904.02689), [CentripetalNet](https://arxiv.org/abs/2003.09119).
- Add more documentations for easier and more clear usage.

Backwards Incompatible Changes

**FP16 related methods are imported from mmcv instead of mmdet. (3766, 3822)**
Mixed precision training utils in `mmdet.core.fp16` are moved to `mmcv.runner`, including `force_fp32`, `auto_fp16`, `wrap_fp16_model`, and `Fp16OptimizerHook`. A deprecation warning will be raised if users attempt to import those methods from `mmdet.core.fp16`, and will be finally removed in V2.8.0.

**[0, N-1] represents foreground classes and N indicates background classes for all models. (3221)**
Before v2.5.0, the background label for RPN is 0, and N for other heads. Now the behavior is consistent for all models. Thus `self.background_labels` in `dense_heads` is removed and all heads use `self.num_classes` to indicate the class index of background labels.
This change has no effect on the pre-trained models in the v2.x model zoo, but will affect the training of all models with RPN heads. Two-stage detectors whose RPN head uses softmax will be affected because the order of categories is changed.

**Only call `get_subset_by_classes` when `test_mode=True` and `self.filter_empty_gt=True` (3695)**
Function `get_subset_by_classes` in dataset is refactored and only filters out images when `test_mode=True` and `self.filter_empty_gt=True`.
In the original implementation, `get_subset_by_classes` is not related to the flag `self.filter_empty_gt` and will only be called when the classes is set during initialization no matter `test_mode` is `True` or `False`. This brings ambiguous behavior and potential bugs in many cases. After v2.5.0, if `filter_empty_gt=False`, no matter whether the classes are specified in a dataset, the dataset will use all the images in the annotations. If `filter_empty_gt=True` and `test_mode=True`, no matter whether the classes are specified, the dataset will call ``get_subset_by_classes` to check the images and filter out images containing no GT boxes. Therefore, the users should be responsible for the data filtering/cleaning process for the test dataset.

New Features

- Test time augmentation for single stage detectors (3844, 3638)
- Support to show the name of experiments during training (3764)
- Add `Shear`, `Rotate`, `Translate` Augmentation (3656, 3619, 3687)
- Add image-only transformations including `Constrast`, `Equalize`, `Color`, and `Brightness`. (3643)
- Support [YOLACT](https://arxiv.org/abs/1904.02689) (#3456)
- Support [CentripetalNet](https://arxiv.org/abs/2003.09119) (#3390)
- Support PyTorch 1.6 in docker (3905)

Bug Fixes

- Fix the bug of training ATSS when there is no ground truth boxes (3702)
- Fix the bug of using Focal Loss when there is `num_pos` is 0 (3702)
- Fix the label index mapping in dataset browser (3708)
- Fix Mask R-CNN training stuck problem when ther is no positive rois (3713)
- Fix the bug of `self.rpn_head.test_cfg` in `RPNTestMixin` by using `self.rpn_head` in rpn head (3808)
- Fix deprecated `Conv2d` from mmcv.ops (3791)
- Fix device bug in RepPoints (3836)
- Fix SABL validating bug (3849)
- Use `https://download.openmmlab.com/mmcv/dist/index.html` for installing MMCV (#3840)
- Fix nonzero in NMS for PyTorch 1.6.0 (3867)
- Fix the API change bug of PAA (3883)
- Fix typo in bbox_flip (3886)
- Fix cv2 import error of ligGL.so.1 in Dockerfile (3891)

Improvements

- Change to use `mmcv.utils.collect_env` for collecting environment information to avoid duplicate codes (3779)
- Update checkpoint file names to v2.0 models in documentation (3795)
- Update tutorials for changing runtime settings (3778), modifing loss (3777)
- Improve the function of `simple_test_bboxes` in SABL (3853)
- Convert mask to bool before using it as img's index for robustness and speedup (3870)
- Improve documentation of modules and dataset customization (3821)

New Contributors
* jd730 made their first contribution in https://github.com/open-mmlab/mmdetection/pull/3741
* yl-1993 made their first contribution in https://github.com/open-mmlab/mmdetection/pull/3840
* chongzhou96 made their first contribution in https://github.com/open-mmlab/mmdetection/pull/3456
* LabMemNo003 made their first contribution in https://github.com/open-mmlab/mmdetection/pull/3876

**Full Changelog**: https://github.com/open-mmlab/mmdetection/compare/v2.4.0...v2.5.0

Highlights
- Fix lots of issues/bugs and reorganize the trouble shooting page
- Support new methods [SABL](https://arxiv.org/abs/1912.04260), [YOLOv3](https://arxiv.org/abs/1804.02767), and [PAA Assign](https://arxiv.org/abs/2007.08103)
- Support Batch Inference
- Start to publish `mmdet` package to PyPI since v2.3.0
- Switch model zoo to download.openmmlab.com

Backwards Incompatible Changes
- Support Batch Inference (3564, 3686, 3705): Since v2.4.0, MMDetection could inference model with multiple images in a single GPU.
This change influences all the test APIs in MMDetection and downstream codebases. To help the users migrate their code, we use `replace_ImageToTensor` (3686) to convert legacy test data pipelines during dataset initialization.
- Support RandomFlip with horizontal/vertical/diagonal direction (3608): Since v2.4.0, MMDetection supports horizontal/vertical/diagonal flip in the data augmentation. This influences bounding box, mask, and image transformations in data augmentation process and the process that will map those data back to the original format.
- Migrate to use `mmlvis` and `mmpycocotools` for COCO and LVIS dataset (3727). The APIs are fully compatible with the original `lvis` and `pycocotools`. Users need to uninstall the existing pycocotools and lvis packages in their environment first and install `mmlvis` & `mmpycocotools`.

Bug Fixes
- Fix default mean/std for onnx (3491)
- Fix coco evaluation and add metric items (3497)
- Fix typo for install.md (3516)
- Fix atss when sampler per gpu is 1 (3528)
- Fix import of fuse_conv_bn (3529)
- Fix bug of gaussian_target, update unittest of heatmap (3543)
- Fixed VOC2012 evaluate (3553)
- Fix scale factor bug of rescale (3566)
- Fix with_xxx_attributes in base detector (3567)
- Fix boxes scaling when number is 0 (3575)
- Fix rfp check when neck config is a list (3591)
- Fix import of fuse conv bn in benchmark.py (3606)
- Fix webcam demo (3634)
- Fix typo and itemize issues in tutorial (3658)
- Fix error in distributed training when some levels of FPN are not assigned with bounding boxes (3670)
- Fix the width and height orders of stride in valid flag generation (3685)
- Fix weight initialization bug in Res2Net DCN (3714)
- Fix bug in OHEMSampler (3677)

New Features
- Support Cutout augmentation (3521)
- Support evaluation on multiple datasets through ConcatDataset (3522)
- Support [PAA assign](https://arxiv.org/abs/2007.08103) #(3547)
- Support eval metric with pickle results (3607)
- Support [YOLOv3](https://arxiv.org/abs/1804.02767) (#3083)
- Support [SABL](https://arxiv.org/abs/1912.04260) (#3603)
- Support to publish to Pypi in github-action (3510)
- Support custom imports (3641)

Improvements
- Refactor common issues in documentation (3530)
- Add pytorch 1.6 to CI config (3532)
- Add config to runner meta (3534)
- Add eval-option flag for testing (3537)
- Add init_eval to evaluation hook (3550)
- Add include_bkg in ClassBalancedDataset (3577)
- Using config's loading in inference_detector (3611)
- Add ATSS ResNet-101 models in model zoo (3639)
- Update urls to download.openmmlab.com (3665)
- Support non-mask training for CocoDataset (3711)

New Contributors
* asuith made their first contribution in https://github.com/open-mmlab/mmdetection/pull/3516
* mprzewie made their first contribution in https://github.com/open-mmlab/mmdetection/pull/3575

**Full Changelog**: https://github.com/open-mmlab/mmdetection/compare/v2.3.0...v2.4.0