Openmixup

Support new features as [6](https://github.com/Westlake-AI/openmixup/issues/6).

New Features

- Support the [online document](https://westlake-ai.github.io/openmixup/) of OpenMixup (built on Read the Docs).
- Provide README and update configs for [self-supervised](https://github.com/Westlake-AI/openmixup/tree/main/configs/selfsup/) and [supervised](https://github.com/Westlake-AI/openmixup/tree/main/configs/classification/) methods.
- Support new Masked Image Modeling (MIM) methods ([A2MIM](https://arxiv.org/abs/2205.13943), [CAE](https://arxiv.org/abs/2202.03026)).
- Support new backbone networks ([DenseNet](https://arxiv.org/abs/1608.06993), [ResNeSt](https://arxiv.org/abs/2004.08955), [PoolFormer](https://arxiv.org/abs/2111.11418), [UniFormer](https://arxiv.org/abs/2201.09450)).
- Support new Fine-tuing method ([HCR](https://arxiv.org/abs/2206.00845)).
- Support new mixup augmentation methods ([SmoothMix](https://openaccess.thecvf.com/content_CVPRW_2020/papers/w45/Lee_SmoothMix_A_Simple_Yet_Effective_Data_Augmentation_to_Train_Robust_CVPRW_2020_paper.pdf), [GridMix](https://www.sciencedirect.com/science/article/pii/S0031320320303976)).
- Support more regression losses ([Focal L1/L2 loss](https://arxiv.org/abs/2102.09554), [Balanced L1 loss](https://arxiv.org/abs/1904.02701), [Balanced MSE loss](https://arxiv.org/abs/2203.16427)).
- Support more regression metrics (regression errors and correlations) and the regression dataset.
- Support more reweight classification losses ([Gradient Harmonized loss](https://arxiv.org/abs/1811.05181), [Varifocal Focal Loss](https://arxiv.org/abs/1811.05181)) from [MMDetection](https://github.com/open-mmlab/mmdetection).

Bug Fixes

- Refactor code structures of `openmixup.models.utils` and support more network layers.
- Fix the bug of `DropPath` (using stochastic depth rule) in `ResNet` for RSB A1/A2 training settings.

Support new features and finish code refactoring as [5](https://github.com/Westlake-AI/openmixup/issues/5).

Highlight

- Support more self-supervised methods ([Barlow Twins](https://arxiv.org/abs/2103.03230) and Masked Image Modeling methods).
- Support popular backbones ([ConvMixer](https://arxiv.org/abs/2201.09792), [MLPMixer](https://arxiv.org/abs/2105.01601), [VAN](https://arxiv.org/abs/2202.09741)) based on MMClassification.
- Support more regression losses ([Charbonnier loss](https://arxiv.org/abs/1710.01992v1) and [Focal Frequency loss](https://arxiv.org/pdf/2012.12821.pdf)).

Bug Fixes

- Fix bugs in self-supervised classification benchmarks (configs and implementations of VisionTransformer).
- Update [INSTALL.md](INSTALL.md). We suggest you install **PyTorch 1.8** or higher and mmcv-full for better usage of this repo. **PyTorch 1.8** has bugs in AdamW optimizer (do not use **PyTorch 1.8** to fine-tune ViT-based methods).
- Fix bugs in PreciseBNHook (update all BN stats) and RepeatSampler (set sync_random_seed).

Support new features and finish code refactoring as [4](https://github.com/Westlake-AI/openmixup/issues/4).

New Features

- Support masked image modeling (MIM) self-supervised methods ([MAE](https://arxiv.org/abs/2111.06377), [SimMIM](https://arxiv.org/abs/2111.09886), [MaskFeat](https://arxiv.org/abs/2112.09133)).
- Support visualization of reconstruction results in MIM methods.
- Support basic regression losses and metrics.

Bug Fixes

- Fix bugs in regression metrics, MIM dataset, and benchmark configs. Notice that only `l1_loss` is supported by FP16 training, other regression losses (e.g., MSE and Smooth_L1 losses) will cause NAN when the target and prediction are not normalized in FP16 training.
- We suggest you install **PyTorch 1.8** or higher (required by some self-supervised methods) and `mmcv-full` for better usage of this repo. Do not use **PyTorch 1.8** to fine-tune ViT-based methods, and you can still use **PyTorch 1.6** for supervised classification methods.

Support new features and finish code refactoring as [3](https://github.com/Westlake-AI/openmixup/issues/3).

New Features

- Support various popular backbones (ConvNets and ViTs), various image datasets, popular mixup methods, and benchmarks for supervised learning. Config files are available.
- Support popular self-supervised methods (e.g., BYOL, MoCo.V3, MAE) on both large-scale and small-scale datasets, and self-supervised benchmarks (merged from MMSelfSup). Config files are available.
- Support analyzing tools for self-supervised learning (kNN/SVM/linear metrics and t-SNE/UMAP visualization).
- Convenient usage of configs: fast configs generation by 'auto_train.py' and configs inheriting (MMCV).
- Support mixed-precision training (NVIDIA Apex or MMCV Apex) for all methods.
- [Model Zoos](docs/model_zoos) and lists of [Awesome Mixups](docs/awesome_mixups) have been released.

Bug Fixes

- Done code refactoring follows MMSelfSup and MMClassification.

- Refactor code structures for vision transformers and self-supervised methods (e.g., [MoCo.V3](https://arxiv.org/abs/2104.02057) and [MAE](https://arxiv.org/abs/2111.06377)).
- Provide online analysis of self-supervised methods (knn metric and t-SNE/UMAP visualization).
- More results are provided in Model Zoos.

Bug Fixes

- Fix bugs of reusing of configs, ViTs, visualization tools, etc. It requires rebuilding of OpenMixup (install mmcv-full).

New Features

- Refactor code structures according to MMSelfsup to fit high version of mmcv and PyTorch.
- Support self-supervised methods and optimizes config structures.