We would like to thank [_Piotr Dollar_](https://github.com/pdollar), [_Mannat Singh_](https://github.com/mannatsingh) and [_Hugo Touvron_](https://github.com/TouvronHugo) for their past research and contributions to this work.
Ops and Transforms
New Augmentations, Layers and Losses
This release brings a bunch of new primitives which can be used to produce SOTA models. Some highlights include the addition of [_AugMix_](https://arxiv.org/abs/1912.02781) data-augmentation method, the [_DropBlock_](https://arxiv.org/abs/1810.12890) layer, the [_cIoU/dIoU_](https://arxiv.org/abs/1911.08287) loss and [_many more_](https://github.com/pytorch/vision/issues/5410). We would like to thank [_Aditya Oke_](https://github.com/oke-aditya), [_Abhijit Deo_](https://github.com/abhi-glitchhg), [_Yassine Alouini_](https://github.com/yassineAlouini) and [_Hu Ye_](https://github.com/xiaohu2015) for contributing to the project and for helping us maintain TorchVision relevant and fresh.
Documentation
We completely revamped our models documentation to make them easier to browse, and added various key information such as supported image sizes, or image pre-processing steps of pre-trained weights. We now have a [main model page](https://pytorch.org/vision/main/models.html) with various [summary tables](https://pytorch.org/vision/main/models.html#table-of-all-available-classification-weights) of available weights, and each model has a [dedicated page](https://pytorch.org/vision/main/models/resnet.html). Each model builder is also documented in their [own page](https://pytorch.org/vision/main/models/generated/torchvision.models.resnet50.html#torchvision.models.resnet50), with more details about the available weights, including accuracy, minimal image size, link to training recipes, and other valuable info. For comparison, our previous models docs are [here](https://pytorch.org/vision/0.12/models.html). To provide feedback on the new documentation, please use the dedicated [Github issue](https://github.com/pytorch/vision/issues/5511).
Backward-incompatible changes
The new Multi-weight support API replaced the legacy “pretrained” parameter of model builders. Both solutions are currently supported to maintain backwards compatibility but our intention is to remove the deprecated API in 2 versions. Migrating to the new API is very straightforward. The following method calls between the 2 APIs are all equivalent:
py
from torchvision.models import resnet50, ResNet50_Weights
Using pretrained weights:
resnet50(weights=ResNet50_Weights.IMAGENET1K_V1)
resnet50(weights="IMAGENET1K_V1")
resnet50(pretrained=True) deprecated
resnet50(True) deprecated
Using no weights:
resnet50(weights=None)
resnet50()
resnet50(pretrained=False) deprecated
resnet50(False) deprecated
Deprecations
[models, models.quantization] Reinstate and deprecate `model_urls` and `quant_model_urls` (5992)
[transforms] Deprecate int as interpolation argument type (5974)
New Features
[models] New Multi-weight API support (5618, 5859, 6047, 6026, 5848)
[models] Adding Swin Transformer architecture (5491)
[models] Adding EfficientNetV2 architecture (5450)
[models] Adding detection model improved weights: RetinaNet, MaskRCNN, FasterRCNN (5756, 5773, 5763)
[models] Adding classification model weight: resnext101 64x4d, mnasnet0_75, mnasnet1_3 (5935, 6019)
[models] Add SWAG model pretrained weights (5714, 5722, 5732, 5793, 5721)
[ops] AddingIoU loss function variants: DIoU, CIoU (5786, 5776)
[ops] Adding various ops and test for ops (6053, 5416, 5792, 5783)
[transforms] Adding AugMix transforms implementation (5411)
[reference scripts] Support custom weight decay setting in classification reference script (5671)
[transforms, reference scripts] Improve detection reference script: Scale Jitter, RandomShortestSize, FixedSizeCrop (5435, 5610, 5607)
[ci] Add M1 support : (6167)
[ci] Add Python-3.10 (build and test) (5420)
Improvements
[documentation] Complete new revamp of models documentation (5821, 5876, 5899, 6025, 5885, 5884, 5886, 5891, 6023, 6009, 5852, 5831, 5832, 6003, 6013, 5856, 6004, 6005, 5878, 6012, 5894, 6002, 5854, 5864, 5920, 5869, 5871, 6021, 6006, 6016, 5905, 6028, 5915, 5924, 5977, 5918, 5921, 5934, 5936, 5937, 5933, 5949, 5988, 5962, 5963, 5975, 5900, 5917, 5895, 5901, 6033, 6032, 6030, 5904, 5661, 6035, 6049, 6036, 5908, 5907, 6044, 6039, 5874, 6151)
[documentation] Various documentation improvements (5695, 5930, 5814, 5799, 5827, 5796, 5923, 5599, 5554, 5995, 5457, 6163, 6031, 6000, 5847, 6024))
[documentation] Add warnings in docs to document Beta APIs (6115)
[datasets] improve GDrive downloads (5704, 5645)
[datasets] indicate md5 checksum is not used for security (5717)
[models] Add shufflenetv2 1.5 and 2.0 weights (5906)
[models] Reduce unnecessary cuda sync in anchor_utils.py (5515)
[models] Adding improved MobileNetV2 weights (5560)
[models] Remove `(N, T, H, W, C) => (N, T, C, H, W)` from presets (6058)
[models] add swin_s and swin_b variants and improved swin_t (6048)
[models] Update ShuffleNetV2 annotations for x1_5 and x2_0 variants (6022)
[models] Better error message in ViT (5820)
[models, ops] Add private support for ciou and diou (5984, 5685, 5690)
[models, reference scripts] Various improvements to detection recipe and models (5715, 5444)
[transforms, tests] add functional vertical flip tests on segmentation mask (5860)
[transforms] make _max_value jit-scriptable (5623)
[transforms] Make ScaleJitter proportional (5559)
[transforms] add tensor kernels for normalize and erase (5462)
[transforms] Update transforms following PIL deprecation (5898)
[transforms, models, datasets…] Replace asserts with exceptions (5587, 5659)
[utils] add warning if font is not set in draw_bounding_boxes (5785)
[utils] Throw warning for empty masks or box tensors on draw_segmentation_masks and draw_bounding_boxes (5857)
[video] Add output_format do video datasets and readers (6061)
[video, io] Better compatibility with FFMPEG 5.0 (5644)
[video, io] Allow cuda device to be passed without the index for GPU decoding (5505)
[reference scripts] Simplify EMA to use Pytorch's update_parameters (5469)
[reference scripts] Reduce variance of evaluation in reference (5819)
[reference scripts] Various improvements to RAFT training reference (5590)
[tests] Speed up Model tests by 20% (5574)
[tests] Make test suite fail on unexpected test success (5556)
[tests] Skip big model in test to reduce memory usage in CI (5903, 5902)
[tests] Improve test of backbone utils (5552)
[tests] Validate against expected files on videos (6077)
[ci] Support for CUDA 11.6 (5803, 5862)
[ci] pre-download model weights in CI docs build (5625)
Bug Fixes
[transforms] remove option to pass fill as str in transforms (5632)
[transforms] Better handling for Pad's fill argument (5596)
[transforms] [FBcode->GH] Fix accimage tests (5545)
[transforms] Update _pil_constants.py (6154) (6156)
[transforms] Fix resize transform when size == small_edge_size and max_size isn't None (5409)
[transforms] Fixed rotate transform with expand inconsistency (5677)
[transforms] Fixed upstream issue with padding (5875)
[transforms] Fix functional.adjust_gamma (5427)
[models] Respect `strict=False` when loading detection models (5841)
[models] Fix resnet norm initialization (6082) (6085)
[models] Use frozen BN only if pre-trained for detection models. (5443)
[models] fix fcos gtarea calculation (5816)
[models, onnx] Add topk min function for trace and onnx (5310)
[models, tests] fix mobilnet norm layer test (5643)
[reference scripts] Fix regression on Detection training script (5985)
[datasets] do not re-download from GDrive if file is already present (5805)
[datasets] Fix datasets: kinetics, Flowers102, VOC_2009, INaturalist 2021_train, caltech (5578, 5775, 5425, 5844, 5789)
[documentation] Fixes device mismatch issue while building docs (5428)
[documentation] Fix Accuracy meta-data on shufflenetv2 (5896)
[documentation] fix typo in docstrings of some transforms (5609)
[video, documentation] Fix append of audio_pts (5488)
[io, tests] More robust check in tests for 16 bits images (5652)
[video, io] Fix shape mismatch error in video reader (5489)
[io] Address nvjpeg leak on CUDA < 11.6 issue (5713, 5482)
[ci] Fixing issue with setup_env.sh in docker: resolve "unsafe directory" error (6106) (6109)
[ci] fix documentation version problems when new release is tagged (5583)
[ci] Replace jcenter and fix version for android (6046)
[tests] Add .float() before .mean() on test_backbone_utils.py because .mean() dont accept integer dtype (6090) (6091)
[tests] Fix keypointrcnn_resnet50_fpn flaky test (5911)
[tests] Disable `test_encode|write_jpeg_reference` tests (5910)
[mobile] Bump up LibTorchvision version number for Podspec to release Cocoapods (5624)
[feature extraction] Add default tracer args for model feature extraction function (5637)
[build] Fix libtorchvision.so not able to encode images by adding *_FOUND macros to CMakeLists.txt (5547)
Code Quality
[dataset, models] Better deprecation message for voc2007 and SqueezeExcitation (5391)
[datasets, reference scripts] Use Kinetics instead of Kinetics400 in references (5787) (5952)
[models] CleanUp DenseNet code (5966)
[models] Minor Swin Transformer fixes (6054)
[models, onnx] Use onnx function only in tracing mode (5468)
[models] Refactor swin transfomer so later we can reuse component for 3d version (6088) (6100)
[models, tests] Fix minor issues with model tests. (5576)
[transforms] Remove `to_tensor()` and `ToTensor()` usages (5553)
[transforms] Refactor Augmentation Space calls to speed up. (5402)
[transforms] Recoded _max_value method using a dictionary (5566)
[transforms] Replace get_image_size/num_channels with get_dimensions (5487)
[ops] Replace usages of atomicAdd with gpuAtomicAdd (5823)
[ops] Fix unused variable warning in ps_roi_align_kernel.cu (5408)
[ops] Remove custom ops interpolation with antialiasing (5329)
[ops] Move Permute layer to ops. (6055)
[ops] Remove assertions for generalized_box_iou (5691)
[utils] Moving `sequence_to_str` to `torchvision._utils` (5604)
[utils] Clarify TypeError message in make_grid (5997)
[video, io] replace distutils.spawn with shutil.which per PEP632 in setup script (5849)
[video, io] Move VideoReader out of __init__ (5495)
[video, io] Remove unnecessary initialisation in GPUDecoder (5507)
[video, io] Remove unused member variable and argument in GPUDecoder (5499)
[video, io] Improve test_video_reader (5498)
[video, io] Update private attribute name for readability (5484)
[video, tests] Improve test_videoapi (5497)
[reference scripts] Minor updates to optical flow ref for consistency (5654)
[reference scripts] Add barrier() after init_process_group() (5475)
[ci] Delete stale packaging scripts (5433)
[ci] remove explicit install of Pillow throughout CI (5950)
[ci, test] remove unnecessary pytest install (5739)
[ci, tests] Remove unnecessary PYTORCH_TEST_WITH_SLOW env (5631)
[ci] Add .git-blame-ignore-revs to ignore specific commits in git blame (5696)
[ci] Remove CUDA 11.1 support (5477, 5470, 5451, 5978)
[ci] Minor linting improvement (5880)
[ci] Remove Bandit and CodeQL jobs (5734)
[ci] Various type annotation fixes / issues (5598, 5970, 5943)
Contributors
We're grateful for our community, which helps us improving torchvision by submitting issues and PRs, and providing feedback and suggestions. The following persons have contributed patches for this release:
Abhijit Deo, Aditya Oke, Andrey Talman, Anton Thomma, Behrooz, Bruno Korbar, Daniel Angelov, Dbhasin1, Drishti Bhasin, F-G Fernandez, Federico Pozzi, FG Fernandez, Georg Grab, Gouvernathor, Hu Ye, Jeffery (Zeyu) Zhao, Joao Gomes, kaijieshi, Kazuki Adachi, KyleCZH, kylematoba, LEGRAND Matthieu, Lezwon Castelino, Luming Tang, Matti Picus, Nicolas Hug, Nikita, Nikita Shulga, oxabz, Philip Meier, Prabhat Roy, puhuk, Richard Barnes, Sahil Goyal, satojkovic, Shijie, Shubham Bhokare, talregev, tcmyxc, Vasilis Vryniotis, vfdev, WuZhe, XiaobingZhang, Xu Zhao, Yassine Alouini, Yonghye Kwon, YosuaMichael, Yulv-git, Zhiqiang Wang