Added
- Added `DataType`, `AverageMethod` and `MDMCAverageMethod` enum in metrics (5657)
- Added support for summarized model total params size in megabytes (5590)
- Added support for multiple train loaders (1959)
- Added `Accuracy` metric now generalizes to Top-k accuracy for (multi-dimensional) multi-class inputs using the `top_k` parameter (4838)
- Added `Accuracy` metric now enables the computation of subset accuracy for multi-label or multi-dimensional multi-class inputs with the `subset_accuracy` parameter (4838)
- Added `HammingDistance` metric to compute the hamming distance (loss) (4838)
- Added `max_fpr` parameter to `auroc` metric for computing partial auroc metric (3790)
- Added `StatScores` metric to compute the number of true positives, false positives, true negatives and false negatives (4839)
- Added `R2Score` metric (5241)
- Added `LambdaCallback` (5347)
- Added `BackboneLambdaFinetuningCallback` (5377)
- Accelerator `all_gather` supports collection (5221)
- Added `image_gradients` functional metric to compute the image gradients of a given input image. (5056)
- Added `MetricCollection` (4318)
- Added `.clone()` method to metrics (4318)
- Added `IoU` class interface (4704)
- Support to tie weights after moving model to TPU via `on_post_move_to_device` hook
- Added missing val/test hooks in `LightningModule` (5467)
- The `Recall` and `Precision` metrics (and their functional counterparts `recall` and `precision`) can now be generalized to RecallK and PrecisionK with the use of `top_k` parameter (4842)
- Added `ModelPruning` Callback (5618, 5825, 6045)
- Added `PyTorchProfiler` (5560)
- Added compositional metrics (5464)
- Added Trainer method `predict(...)` for high performence predictions (5579)
- Added `on_before_batch_transfer` and `on_after_batch_transfer` data hooks (3671)
- Added AUC/AUROC class interface (5479)
- Added `PredictLoop` object (5752)
- Added `QuantizationAwareTraining` callback (5706, 6040)
- Added `LightningModule.configure_callbacks` to enable the definition of model-specific callbacks (5621)
- Added `dim` to `PSNR` metric for mean-squared-error reduction (5957)
- Added promxial policy optimization template to pl_examples (5394)
- Added `log_graph` to `CometLogger` (5295)
- Added possibility for nested loaders (5404)
- Added `sync_step` to Wandb logger (5351)
- Added `StochasticWeightAveraging` callback (5640)
- Added `LightningDataModule.from_datasets(...)` (5133)
- Added `PL_TORCH_DISTRIBUTED_BACKEND` env variable to select backend (5981)
- Added `Trainer` flag to activate Stochastic Weight Averaging (SWA) `Trainer(stochastic_weight_avg=True)` (6038)
- Added DeepSpeed integration (5954, 6042)
Changed
- Changed `stat_scores` metric now calculates stat scores over all classes and gains new parameters, in line with the new `StatScores` metric (4839)
- Changed `computer_vision_fine_tunning` example to use `BackboneLambdaFinetuningCallback` (5377)
- Changed `automatic casting` for LoggerConnector `metrics` (5218)
- Changed `iou` [func] to allow float input (4704)
- Metric `compute()` method will no longer automatically call `reset()` ([5409](https://github.com/PyTorchLightning/pytorch-lightning/pull/5409/))
- Set PyTorch 1.4 as min requirements, also for testing and examples `torchvision>=0.5` and `torchtext>=0.5` (5418)
- Changed `callbacks` argument in `Trainer` to allow `Callback` input (5446)
- Changed the default of `find_unused_parameters` to `False` in DDP (5185)
- Changed `ModelCheckpoint` version suffixes to start at 1 (5008)
- Progress bar metrics tensors are now converted to float (5692)
- Changed the default value for the `progress_bar_refresh_rate` Trainer argument in Google COLAB notebooks to 20 (5516)
- Extended support for purely iteration-based training (5726)
- Made `LightningModule.global_rank`, `LightningModule.local_rank` and `LightningModule.logger` read-only properties (5730)
- Forced `ModelCheckpoint` callbacks to run after all others to guarantee all states are saved to the checkpoint (5731)
- Refactored Accelerators and Plugins (5743)
* Added base classes for plugins (5715)
* Added parallel plugins for DP, DDP, DDPSpawn, DDP2 and Horovod (5714)
* Precision Plugins (5718)
* Added new Accelerators for CPU, GPU and TPU (5719)
* Added Plugins for TPU training (5719)
* Added RPC and Sharded plugins (5732)
* Added missing `LightningModule`-wrapper logic to new plugins and accelerator (5734)
* Moved device-specific teardown logic from training loop to accelerator (5973)
* Moved accelerator_connector.py to the connectors subfolder (6033)
* Trainer only references accelerator (6039)
* Made parallel devices optional across all plugins (6051)
* Cleaning (5948, 5949, 5950)
- Enabled `self.log` in callbacks (5094)
- Renamed xxx_AVAILABLE as protected (5082)
- Unified module names in Utils (5199)
- Separated utils: imports & enums (5256, 5874)
- Refactor: clean trainer device & distributed getters (5300)
- Simplified training phase as LightningEnum (5419)
- Updated metrics to use LightningEnum (5689)
- Changed the seq of `on_train_batch_end`, `on_batch_end` & `on_train_epoch_end`, `on_epoch_end hooks` (5688)
- Refactored `setup_training` and remove `test_mode` (5388)
- Disabled training with zero `num_training_batches` when insufficient `limit_train_batches` (5703)
- Refactored `EpochResultStore` (5522)
- Update `lr_finder` to check for attribute if not running `fast_dev_run` (5990)
- LightningOptimizer manual optimizer is more flexible and expose `toggle_model` (5771)
- `MlflowLogger` limit parameter value length to 250 char (5893)
- Re-introduced fix for Hydra directory sync with multiple process (5993)
Deprecated
- Function `stat_scores_multiple_classes` is deprecated in favor of `stat_scores` (4839)
- Moved accelerators and plugins to its `legacy` pkg (5645)
- Deprecated `LightningDistributedDataParallel` in favor of new wrapper module `LightningDistributedModule` (5185)
- Deprecated `LightningDataParallel` in favor of new wrapper module `LightningParallelModule` (5670)
- Renamed utils modules (5199)
* `argparse_utils` >> `argparse`
* `model_utils` >> `model_helpers`
* `warning_utils` >> `warnings`
* `xla_device_utils` >> `xla_device`
- Deprecated using `'val_loss'` to set the `ModelCheckpoint` monitor (6012)
- Deprecated `.get_model()` with explicit `.lightning_module` property (6035)
- Deprecated Trainer attribute `accelerator_backend` in favor of `accelerator` (6034)
Removed
- Removed deprecated checkpoint argument `filepath` (5321)
- Removed deprecated `Fbeta`, `f1_score` and `fbeta_score` metrics (5322)
- Removed deprecated `TrainResult` (5323)
- Removed deprecated `EvalResult` (5633)
- Removed `LoggerStages` (5673)
Fixed
- Fixed distributed setting and `ddp_cpu` only with `num_processes>1` (5297)
- Fixed the saved filename in `ModelCheckpoint` when it already exists (4861)
- Fixed `DDPHPCAccelerator` hangs in DDP construction by calling `init_device` (5157)
- Fixed `num_workers` for Windows example (5375)
- Fixed loading yaml (5619)
- Fixed support custom DataLoader with DDP if they can be re-instantiated (5745)
- Fixed repeated `.fit()` calls ignore max_steps iteration bound (5936)
- Fixed throwing `MisconfigurationError` on unknown mode (5255)
- Resolve bug with Finetuning (5744)
- Fixed `ModelCheckpoint` race condition in file existence check (5155)
- Fixed some compatibility with PyTorch 1.8 (5864)
- Fixed forward cache (5895)
- Fixed recursive detach of tensors to CPU (6007)
- Fixed passing wrong strings for scheduler interval doesn't throw an error (5923)
- Fixed wrong `requires_grad` state after `return None` with multiple optimizers (5738)
- Fixed add `on_epoch_end` hook at the end of `validation`, `test` epoch (5986)
- Fixed missing `process_dataloader` call for `TPUSpawn` when in distributed mode (6015)
- Fixed progress bar flickering by appending 0 to floats/strings (6009)
- Fixed synchronization issues with TPU training (6027)
- Fixed `hparams.yaml` saved twice when using `TensorBoardLogger` (5953)
- Fixed basic examples (5912, 5985)
- Fixed `fairscale` compatible with PT 1.8 (5996)
- Ensured `process_dataloader` is called when `tpu_cores > 1` to use Parallel DataLoader (6015)
- Attempted SLURM auto resume call when non-shell call fails (6002)
- Fixed wrapping optimizers upon assignment (6006)
- Fixed allowing hashing of metrics with lists in their state (5939)
Contributors
alanhdu, ananthsub, awaelchli, Borda, borisdayma, carmocca, ddrevicky, deng-cy, ducthienbui97, justusschock, kartik4949, kaushikb11, manipopopo, marload, neighthan, peblair, prampey, pranjaldatta, rohitgr7, SeanNaren, sid-sundrani, SkafteNicki, tadejsv, tchaton, teddykoker, titu1994, yuntai
_If we forgot someone due to not matching commit email with GitHub account, let us know :]_