- Highlights
- New Features
- Improvements
- Bug Fixes
- Docs Update
Highlights
We are excited to announce that MMClassification and MMSelfSup have been merged into ONE codebase, named MMPreTrain, which has the following highlights:
- Integrated Self-supervised learning algorithms from **MMSelfSup**, such as **MAE**, **BEiT**, etc. Users could find that in our directory `mmpretrain/models`, where a new folder `selfsup` was made, which support 18 recent self-supervised learning algorithms.
| Contrastive leanrning | Masked image modeling |
| :----------------------------: | :----------------------------------: |
| MoCo series | BEiT series |
| SimCLR | MAE |
| BYOL | SimMIM |
| SwAV | MaskFeat |
| DenseCL | CAE |
| SimSiam | MILAN |
| BarlowTwins | EVA |
| DenseCL | MixMIM |
- Support **RIFormer**, which is a way to keep a vision backbone effective while removing token mixers in its basic building blocks. Equipped with our proposed optimization strategy, we are able to build an extremely simple vision backbone with encouraging performance, while enjoying high efficiency during inference.
<div>
<img src="https://user-images.githubusercontent.com/48375204/223930120-dc075c8e-0513-42eb-9830-469a45c1d941.png" width="80%"/>
</div>
- Support **LeViT**, **XCiT**, **ViG**, and **ConvNeXt-V2** backbone, thus currently we support 68 backbones or algorithms and 472 checkpoints.
- Add t-SNE visualization, users could visualize t-SNE to analyze the ability of your backbone. An example of visualization: left is from `MoCoV2_ResNet50` and the right is from `MAE_ViT-base`:
<div>
<img src="https://user-images.githubusercontent.com/36138628/207305086-91df298c-0eb7-4254-9c5b-ba711644501b.png" width="40%" />
<img src="https://user-images.githubusercontent.com/36138628/223383663-a021bb5f-1ef5-404d-87aa-c353edd4e1e1.png" width="40%" />
</div>
- Refactor dataset pipeline visualization, now we could also visualize the pipeline of mask image modeling, such as BEiT:
<div><img src="https://user-images.githubusercontent.com/26739999/226542300-74216187-e3d0-4a6e-8731-342abe719721.png" width="70%"></div>
New Features
- Support RIFormer. ([1453](https://github.com/open-mmlab/mmpretrain/pull/1453))
- Support XCiT Backbone. ([1305](https://github.com/open-mmlab/mmclassification/pull/1305))
- Support calculate confusion matrix and plot it. ([1287](https://github.com/open-mmlab/mmclassification/pull/1287))
- Support RetrieverRecall metric & Add ArcFace config ([1316](https://github.com/open-mmlab/mmclassification/pull/1316))
- Add `ImageClassificationInferencer`. ([1261](https://github.com/open-mmlab/mmclassification/pull/1261))
- Support InShop Dataset (Image Retrieval). ([1019](https://github.com/open-mmlab/mmclassification/pull/1019))
- Support LeViT backbone. ([1238](https://github.com/open-mmlab/mmclassification/pull/1238))
- Support VIG Backbone. ([1304](https://github.com/open-mmlab/mmclassification/pull/1304))
- Support ConvNeXt-V2 backbone. ([1294](https://github.com/open-mmlab/mmclassification/pull/1294))
Improvements
- Use PyTorch official `scaled_dot_product_attention` to accelerate `MultiheadAttention`. ([1434](https://github.com/open-mmlab/mmpretrain/pull/1434))
- Add ln to vit avg_featmap output ([1447](https://github.com/open-mmlab/mmpretrain/pull/1447))
- Update analysis tools and documentations. ([1359](https://github.com/open-mmlab/mmclassification/pull/1359))
- Unify the `--out` and `--dump` in `tools/test.py`. ([1307](https://github.com/open-mmlab/mmclassification/pull/1307))
- Enable to toggle whether Gem Pooling is trainable or not. ([1246](https://github.com/open-mmlab/mmclassification/pull/1246))
- Update registries of mmcls. ([1306](https://github.com/open-mmlab/mmclassification/pull/1306))
- Add metafile fill and validation tools. ([1297](https://github.com/open-mmlab/mmclassification/pull/1297))
- Remove useless EfficientnetV2 config files. ([1300](https://github.com/open-mmlab/mmclassification/pull/1300))
Bug Fixes
- Fix precise bn hook ([1466](https://github.com/open-mmlab/mmpretrain/pull/1466))
- Fix retrieval multi gpu bug ([1319](https://github.com/open-mmlab/mmclassification/pull/1319))
- Fix error repvgg-deploy base config path. ([1357](https://github.com/open-mmlab/mmclassification/pull/1357))
- Fix bug in test tools. ([1309](https://github.com/open-mmlab/mmclassification/pull/1309))
Docs Update
- Translate some tools tutorials to Chinese. ([1321](https://github.com/open-mmlab/mmclassification/pull/1321))
- Add Chinese translation for runtime.md. ([1313](https://github.com/open-mmlab/mmclassification/pull/1313))
Contributors
A total of 13 developers contributed to this release.
Thanks to techmonsterwang , qingtian5 , mzr1996 , okotaku , zzc98 , aso538 , szwlh-c , fangyixiao18 , yukkyo , Ezra-Yu , csatsurnh , 2546025323 , GhaSiKey .
New Contributors
* csatsurnh made their first contribution in https://github.com/open-mmlab/mmpretrain/pull/1309
* szwlh-c made their first contribution in https://github.com/open-mmlab/mmpretrain/pull/1304
* aso538 made their first contribution in https://github.com/open-mmlab/mmpretrain/pull/1238
* GhaSiKey made their first contribution in https://github.com/open-mmlab/mmpretrain/pull/1313
* yukkyo made their first contribution in https://github.com/open-mmlab/mmpretrain/pull/1246
* 2546025323 made their first contribution in https://github.com/open-mmlab/mmpretrain/pull/1321
**Full Changelog**: https://github.com/open-mmlab/mmpretrain/compare/v1.0.0rc5...v1.0.0rc7