Catalyst

Added

- `catalyst-run` for Config API support added [1406](https://github.com/catalyst-team/catalyst/pull/1406)


Fixed

- Logger API naming [1405](https://github.com/catalyst-team/catalyst/pull/1405)

Tl;dr
- Catalyst architecture simplification.
- [1395](https://github.com/catalyst-team/catalyst/issues/1395), [#1396](https://github.com/catalyst-team/catalyst/issues/1396), [#1397](https://github.com/catalyst-team/catalyst/issues/1397), [#1398](https://github.com/catalyst-team/catalyst/issues/1398), [#1399](https://github.com/catalyst-team/catalyst/issues/1399), [#1400](https://github.com/catalyst-team/catalyst/issues/1400), [#1401](https://github.com/catalyst-team/catalyst/issues/1401), [#1402](https://github.com/catalyst-team/catalyst/issues/1402), [#1403](https://github.com/catalyst-team/catalyst/issues/1403).

Added

- Additional tests for different hardware accelerators setups. Please check out the `tests/pipelines` folder for more information.
- `BackwardCallback` and `BackwardCallbackOrder` as an abstraction on top of `loss.backward`. Now you could easily log model gradients or transform them before `OptimizerCallback`.
- `CheckpointCallbackOrder` for `ICheckpointCallback`.

Changed

- Minimal python version moved to `3.7`, minimal pytorch version moved to `1.4.0`.
- Engines rewritten on top of Accelerate. First, we found these two abstractions very close to each other. Second, Accelerate provides additional user-friendly API and more stable API for "Nvidia APEX" and "Facebook Fairscale" - it does not support them.
- SelfSupervisedRunner moved to the `examples` folder from the Catalyst API. The only Runners API, that will be supported in the future: `IRunner`, `Runner`, `ISupervisedRunner`, `SupervisedRunner` due to their consistency. If you are interested in any other Runner API - feel free to write your own `CustomRunner` and use `SelfSupervisedRunner` as an example.
- `Runner.{global/stage}_{batch/loader/epoch}_metrics` renamed to `Runner.{batch/loader/epoch}_metrics`
- `CheckpointCallback` rewritten from scratch.
- Catalyst registry moved to full-imports-paths only.
- Logger API changed to receive `IRunner` for all `log_*` methods.
- Metric API: `topk_args` renamed to `topk`
- Contrib API: init imports from `catalyst.contrib` - removed, use `from catalyst.contrib.{smth} import {smth}`. Could be change to full-imports-only in future versions for stability.
- All quickstarts, minimal examples, notebooks and pipelines moved to new version.
- Codestyle moved to `89` right margin. Honestly speaking, it's much easier to maintain Catalyst with `89` right margin on MBP'16.

Removed

- `ITrial` removed.
- Stages support removed. While we embrace stages in deep learning experiments, current hardware accelerators are not prepared well for such setups. Additionally, ~95% of dl pipelines are single-stage. Multi-stage runner support is under review. For multi-stage support, please define a `CustomRunner` with rewritten API.
- Config/Hydra API support removed. Config API is under review. For now, you could write your own Config API with [hydra-slayer](https://github.com/catalyst-team/hydra-slayer) if needed.
- `catalyst-dl` scripts removed. Without Config API we don't need them anymore.
- `Nvidia Apex`, `Fairscale`, `Albumentations`, `Nifti`, `Hydra` requiremets removed.
- `OnnxCallback`, `PruningCallback`, `QuantizationCallback`, `TracingCallback` removed from callbacks API. Theese callbacks are under review now.

If you have any questions on the Catalyst 22 edition updates, please join Catalyst slack for discussion.

Added

- MNIST dataset for SSL banchmark ([1368](https://github.com/catalyst-team/catalyst/pull/1368))
- MoveiLens 20M dataset [1336](https://github.com/catalyst-team/catalyst/pull/1336)
- logger property for logging customization ([1372](https://github.com/catalyst-team/catalyst/pull/1372))
- MacridVAE example ([1363](https://github.com/catalyst-team/catalyst/pull/1363))
- SSL benchmark results ([1374](https://github.com/catalyst-team/catalyst/pull/1374))
- Neptune example ([1377](https://github.com/catalyst-team/catalyst/pull/1377))
- multi-node support for engines ([1364](https://github.com/catalyst-team/catalyst/pull/1364))

Changed

- RL examples update to last version ([1370](https://github.com/catalyst-team/catalyst/pull/1370))
- DDPLoaderWrapper updated to new version ([1385](https://github.com/catalyst-team/catalyst/pull/1385))
- `num_classes` for classification metrics became optional ([1379](https://github.com/catalyst-team/catalyst/pull/1379))
- colab ci/cd update to new verion

Removed

-

Fixed

- `requests` requirements for `catalyst[cv]` added ([1371](https://github.com/catalyst-team/catalyst/pull/1370))
- loader step counter ([1374](https://github.com/catalyst-team/catalyst/pull/1374))
- detection example data preprocessing ([1369](https://github.com/catalyst-team/catalyst/pull/1369))
- gradient clipping with fp16 runs ([1378](https://github.com/catalyst-team/catalyst/pull/1378))
- config API fix for DDP runs ([1383](https://github.com/catalyst-team/catalyst/pull/1383))
- checkpoint creation for fp16 engines ([1382](https://github.com/catalyst-team/catalyst/pull/1382))

Added

- MultiVAE RecSys example ([1340](https://github.com/catalyst-team/catalyst/pull/1340))
- Returned `resume` support - resolved [1193](https://github.com/catalyst-team/catalyst/issues/1193) ([#1349](https://github.com/catalyst-team/catalyst/pull/1349))
- Smoothing dice loss to contrib ([1344](https://github.com/catalyst-team/catalyst/pull/1344))
- `profile` flag for `runner.train` ([1348](https://github.com/catalyst-team/catalyst/pull/1348))
- MultiDAE RecSys example ([1356](https://github.com/catalyst-team/catalyst/pull/1356))
- `SETTINGS.log_batch_metrics`, `SETTINGS.log_epoch_metrics`, `SETTINGS.compute_per_class_metrics` for framework-wise Metric & Logger APIs specification ([1357](https://github.com/catalyst-team/catalyst/pull/1357))
- `log_batch_metrics` and `log_epoch_metrics` options for all available Loggers ([1357](https://github.com/catalyst-team/catalyst/pull/1357))
- `compute_per_class_metrics` option for all available multiclass/label metrics ([1357](https://github.com/catalyst-team/catalyst/pull/1357))
- pytorch benchmark script and simplified MNIST ([1360](https://github.com/catalyst-team/catalyst/pull/1360))

Changed

- A few framework simplifications were made ([1346](https://github.com/catalyst-team/catalyst/pull/1346)):
- `catalyst-contrib` scripts reduced to `collect-env` and `project-embeddings` only
- `catalyst-dl` scripts recuded to `run` and `tune` only
- `transforms.` prefix deprecated for Catalyst-based transforms
- `catalyst.tools` moved to `catalyst.extras`
- task-dependent extensions from `catalyst.data` moved to `catalyst.contrib.data`
- `catalyst.data.transforms` moved to `catalyst.contrib.data.transforms`
- `Normalize`, `ToTensor` transforms renamed to `NormalizeImage`, `ImageToTensor`
- metric learning extensions moved to `catalyst.contrib.data`
- `catalyst.contrib` moved to code-as-a-documentation development
- `catalyst[cv]` and `catalyst[ml]` extensions moved to flatten architecture design; examples: `catalyst.contrib.data.dataset_cv`, `catalyst.contrib.data.dataset_ml`
- `catalyst.contrib` moved to flatten architecture design; exampels: `catalyst.contrib.data`, `catalyst.contrib.datasets`, `catalyst.contrib.layers`, `catalyst.contrib.models`, `catalyst.contrib.optimizers`, `catalyst.contrib.schedulers`
- internal functionality moved to `***._misc` modules
- `catalyst.utils.mixup` moved to `catalyst.utils.torch`
- `catalyst.utils.numpy` moved to `catalyst.contrib.utils.numpy`
- default logging logic moved from "batch & epoch" to "epoch"-only to save computation time during logging; to respecify, please use:
- `SETTINGS.log_batch_metrics=True/False` or `os.environ["CATALYST_LOG_BATCH_METRICS"]`
- `SETTINGS.log_epoch_metrics=True/False` or `os.environ["CATALYST_LOG_EPOCH_METRICS"]`
- default metrics computation moved from "per-class & aggregations" to "aggregations"-only to save computation time during logging; to respecify, please use:
- `SETTINGS.compute_per_class_metrics=True/False` or `os.environ["CATALYST_COMPUTE_PER_CLASS_METRICS"]`
- no transformations required for MNIST contrib dataset ([1360](https://github.com/catalyst-team/catalyst/pull/1360)

Removed

- A few framework simplifications were made ([1346](https://github.com/catalyst-team/catalyst/pull/1346)):
- `catalyst.contrib.pandas`
- `catalyst.contrib.parallel`
- `catalyst.contrib.models.cv`
- a few `catalyst.utils.misc` functions
- `catalyst.extras` removed from the public documentation


Fixed

- documentation search error (21.10 only) ([1346](https://github.com/catalyst-team/catalyst/pull/1346))
- docs examples ([1362](https://github.com/catalyst-team/catalyst/pull/1362))
- Self-Supervised benchmark: ([1365](https://github.com/catalyst-team/catalyst/pull/1365)), ([#1361](https://github.com/catalyst-team/catalyst/pull/1361))

Added

- RSquareLoss ([1313](https://github.com/catalyst-team/catalyst/pull/1313))
- Self-Supervised example updates: ([1305](https://github.com/catalyst-team/catalyst/pull/1305)), ([#1322](https://github.com/catalyst-team/catalyst/pull/1322)), ([#1325](https://github.com/catalyst-team/catalyst/pull/1325)), ([#1335](https://github.com/catalyst-team/catalyst/pull/1335))
- Albert training example ([1326](https://github.com/catalyst-team/catalyst/pull/1326))
- YOLO-X (new) detection example and refactoring ([1324](https://github.com/catalyst-team/catalyst/pull/1324))
- `TopKMetric` abstraction ([1330](https://github.com/catalyst-team/catalyst/pull/1330))

Changed

- simlified readme ([1312](https://github.com/catalyst-team/catalyst/pull/1312))
- improved DDP tutorial ([1327](https://github.com/catalyst-team/catalyst/pull/1327))
- `CMCMetric` renamed from `<prefix>cmc<suffix><k>` to `<prefix>cmc<k><suffix>` ([1330](https://github.com/catalyst-team/catalyst/pull/1330))

Removed

-

Fixed

- Zero seed error ([1329](https://github.com/catalyst-team/catalyst/pull/1329))
- updated codestyle issues ([1331](https://github.com/catalyst-team/catalyst/pull/1331))
- TopK metrics: ([1330](https://github.com/catalyst-team/catalyst/pull/1330)), ([#1334](https://github.com/catalyst-team/catalyst/pull/1334)), ([#1339](https://github.com/catalyst-team/catalyst/pull/1339))
- `--expdir` param for `catalyst-dl run` ([1338](https://github.com/catalyst-team/catalyst/pull/1338))
- ControlFlowCallback for distributed setup ([1341](https://github.com/catalyst-team/catalyst/pull/1341))

Added

- CometLogger support ([1283](https://github.com/catalyst-team/catalyst/pull/1283))
- CometLogger examples ([1287](https://github.com/catalyst-team/catalyst/pull/1287))
- XLA docs ([1288](https://github.com/catalyst-team/catalyst/pull/1288))
- Contarstive loss functions: `NTXentLoss` ([1278](https://github.com/catalyst-team/catalyst/pull/1278)), `SupervisedContrastiveLoss` ([#1293](https://github.com/catalyst-team/catalyst/pull/1293))
- Self supervised learning: `ISelfSupervisedRunner`, `SelfSupervisedConfigRunner`, `SelfSupervisedRunner`, `SelfSupervisedDatasetWrapper` ([1278](https://github.com/catalyst-team/catalyst/pull/1278))
- SimCLR example ([1278](https://github.com/catalyst-team/catalyst/pull/1278))
- Superivised Contrastive example ([1293](https://github.com/catalyst-team/catalyst/pull/1293))
- extra warnings for runner-callbacks interaction ([1295](https://github.com/catalyst-team/catalyst/pull/1295))
- `CategoricalRegressionLoss` and `QuantileRegressionLoss` to the `contrib` ([1295](https://github.com/catalyst-team/catalyst/pull/1295))
- R2 score metric ([1274](https://github.com/catalyst-team/catalyst/pull/1274))


Changed
- Improved `WandbLogger` to support artifacts and fix logging steps ([1309](https://github.com/catalyst-team/catalyst/pull/1309))
- full `Runner` cleanup, with callbacks and loaders destruction, moved to `PipelineParallelFairScaleEngine` only ([1295](https://github.com/catalyst-team/catalyst/pull/1295))
- `HuberLoss` renamed to `HuberLossV0` for the PyTorch compatibility ([1295](https://github.com/catalyst-team/catalyst/pull/1295))
- codestyle update ([1298](https://github.com/catalyst-team/catalyst/pull/1298))
- BalanceBatchSampler - deprecated ([1303](https://github.com/catalyst-team/catalyst/pull/1303))

Removed

-

Fixed

- CI/CD ([1292](https://github.com/catalyst-team/catalyst/pull/1292)), ([#1299](https://github.com/catalyst-team/catalyst/pull/1299)), ([#1304](https://github.com/catalyst-team/catalyst/pull/1304)), ([#1306](https://github.com/catalyst-team/catalyst/pull/1306))
- Optuna configs ([1296](https://github.com/catalyst-team/catalyst/pull/1292)), ([#1296](https://github.com/catalyst-team/catalyst/pull/1299))