Pytorch-ignite

New Features

- Added `whitelist` argument to log only desired weights/grads with experiment tracking system handlers: 2550, 2523
- Added `ReduceLROnPlateauScheduler` parameter scheduler: 2449
- Added filename components in `Checkpoint`: 2498
- Added missing args to `ModelCheckpoint`, parity with `Checkpoint`: 2486
- **[BC-breaking]** `LRScheduler` is now attachable to `Events.ITERATION_STARTED`: 2496

Bug fixes

- Fixed `zero_grad` place in `create_supervised_trainer` resulting in grad zero logs: 2560, 2559, 2555, 2547
- Fixed bug in `Checkpoint` when loading a single non-`nn.Module` object: 2487
- Removed warning in DDP if `Metric.reset/update` are not decorated: 2549
- **[BC-breaking]** Fixed SSIM metric implementation and issue with variable batch inputs: 2564, 2563
- `compute` method now returns `float` instead of `torch.Tensor`

Housekeeping (docs, CI, examples, tests, etc)

- 2552, 2543, 2541, 2534, 2531, 2530, 2529, 2528, 2526, 2525, 2521, 2518, 2512, 2509, 2507, 2506, 2497, 2494, 2493, 2490, 2485, 2483, 2477, 2476, 2474, 2473, 2469, 2463, 2461, 2460, 2457, 2454, 2450, 2448, 2446, 2445, 2442, 2440, 2439, 2435, 2433, 2431, 2430, 2428, 2427,

Acknowledgments

🎉 Thanks to our community and all our contributors for the issues, PRs and 🌟 ⭐️ 🌟 !
💯 We really appreciate your implication into the project (in alphabetical order):

Davidportlouis, DevPranjal, Ishan-Kumar2, KevinMusgrave, Moh-Yakoub, asmayer, divo12, gorarakelyan, jreese, leotac, nishantb06, nmcguire101, sadra-barikbin, sayantan1410, sdesrozis, vfdev-5, yuta0821

New Features

- Added data as None option to `Engine.run` (2369)
- Now `Checkpoint.load_objects` can accept `str` and load the checkpoint internally (2305)


Bug fixes

- Fixed issue with `DeterministicEngine.state_dict()` (2412)
- Fixed `EMAHandler` warm-up behaviour (2333)
- Fixed `_compute_nproc_per_node` in case of bad dist configuration (2288)
- Fixed state parameter scheduler to work with `EMAHandler` (2326)
- Fixed a bug on `StateParamScheduler.attach` method (2316)
- Fixed `ClearMLLogger` to retrieve current task before trying to create a new one (2344)
- Added hashing a checkpoint utility: 2272, 2283, 2273
- Fixed config check issue with multi-node spawn method (2424)

Housekeeping (docs, CI, examples, tests, etc)

- Added doctests for docstrings: 2241, 2402, 2400, 2399, 2395, 2394, 2391, 2389, 2384, 2352, 2351, 2349, 2348, 2347, 2346, 2345, 2341, 2340, 2336, 2335, 2332, 2327, 2324, 2323, 2321, 2317, 2311, 2307, 2290, 2284, 2280
- 2420, 2411, 2409, 2404, 2392, 2382, 2380, 2378, 2377, 2374, 2371, 2370, 2365, 2362, 2360, 2359, 2357, 2355, 2334, 2331, 2329, 2308, 2297, 2292, 2285, 2279, 2278, 2277, 2270, 2264, 2261, 2252,


Acknowledgments

🎉 Thanks to our community and all our contributors for the issues, PRs and 🌟 ⭐️ 🌟 !
💯 We really appreciate your implication into the project (in alphabetical order):

Abo7atm, DevPranjal, Eunjnnn, FarehaNousheen, H4dr1en, Ishan-Kumar2, KickItLikeShika, Priyansi, bibhabasumohapatra, fco-dv, louis-she, sandylaker, sdesrozis, trsvchn, vfdev-5, ydcjeff

New Features

- Enabled `LRFinder` to run multiple epochs (2200)
- `save_handler` automatically detects `DiskSaver` when path passed (2198)
- Improved `Checkpoint` to use `score_name` as metric's key (2146)
- Added `State` parameter scheduler (2090)
- Added state attributes for loggers (tqdm, Polyaxon, MLFlow, WandB, Neptune, Tensorboard, Visdom, ClearML) (2162, 2161, 2160, 2154, 2153, 2152, 2151, 2148, 2140, 2137)
- Added gradient accumulation to supervised training step functions (2223)
- Automatic jupyter environment detection (2188)
- Added an additional argument to `auto_optim` to allow gradient accumulation (2169)
- Added micro averaging for Bleu Score (2179)
- Expanded BLEU, ROUGE to be calculated on batch input (2259, 2180)
- Moved `BasicTimeProfiler`, `HandlersTimeProfiler`, `ParamScheduler`, `LRFinder` to core (2136, 2135, 2132)

Bug fixes

- Fixed docstring examples with huge bottom padding (2225)
- Fixed NCCL warning caused by barrier if using idist (2257, 2254)
- Fixed hostname list expansion (2208, 2204)
- Fixed tcp error with PyTorch v1.9.1 (2211)

Housekeeping (docs, CI, examples, tests, etc)

- 2243, 2242, 2228, 2164, 2222, 2221, 2220, 2219, 2218, 2217, 2216, 2173, 2164, 2207, 2236, 2190, 2256, 2196, 2177, 2166, 2155, 2149, 2234, 2206, 2186, 2176, 2246, 2231, 2182, 2192, 2165, 2227, 2253, 2247, 2250, 2226, 2201, 2184, 2142, 2232, 2238, 2174

Acknowledgments

🎉 Thanks to our community and all our contributors for the issues, PRs and 🌟 ⭐️ 🌟 !
💯 We really appreciate your implication into the project (in alphabetical order):

Chandan-h-509, Ishan-Kumar2, KickItLikeShika, Priyansi, fco-dv, gucifer, kennethleungty, logankilpatrick, mfoglio, sandylaker, sdesrozis, theory-in-progress, toxa23, trsvchn, vfdev-5, ydcjeff

New Features

- Added `start_lr` option to `FastaiLRFinder` (2111)
- Added Model's EMA handler (2098, 2102)
- Improved SLURM support: added hostlist expansion without using `scontrol` (2092)

Metrics

- Added Inception Score (2053)
- Added FID metric (2049, 2061, 2085, 2094, 2103)
- Blog post "GAN Evaluation : the Frechet Inception Distance and Inception Score metrics" (https://pytorch-ignite.ai/posts/gan-evaluation-with-fid-and-is/)
- Improved DDP support for metrics (2096, 2083)
- Improved `MetricsLambda` to work with `reset/update/compute` API (2091)

Bug fixes

- Modified `auto_dataloader` to not wrap user provided `DistributedSampler` (2119)
- Raise error in `DistributedProxySampler` when sampler is already a `DistributedSampler` (2120)
- Improved LRFinder error message (2127)
- Added `py.typed` for type checkers (2095)

Housekeeping

- 2123, 2117, 2116, 2110, 2093, 2086

Acknowledgments

🎉 Thanks to our community and all our contributors for the issues, PRs and 🌟 ⭐️ 🌟 !
💯 We really appreciate your implication into the project (in alphabetical order):

01-vyom, KickItLikeShika, gucifer, sandylaker, schuhschuh, sdesrozis, trsvchn, vfdev-5, ydcjeff

New Features

Metrics

- Added BLEU metric (1834)
- Added ROUGE metric (1772)
- Added MultiLabelConfusionMatrix metric (1613)
- Added Cohen Kappa metric (1690)
- Extended `sync_all_reduce` API (1823)
- Made `EpochMetric` more generic by extending the list of valid types (1748)
- Fixed issue with metric's output device (2062)
- Added support for list of tensors as metric input (2055)
- Implemented Jaccard Index shortcut for metrics (1682)
- Updated Loss metric to use `required_output_keys` (2027)
- Added classification report metric (1887)
- Added output detach for Canberra metric (1820)
- Improved ROC AUC (1762)
- Improved AveragePrecision metric and tests (1756)
- Uniformly handling of metric types for all loggers (2021)
- More DDP support for multiple contrib metrics (1891, 1869, 1865, 1850, 1830, 1829, 1806, 1805, 1803)


Engine

- Added native `torch.cuda.amp` and `apex` automatic mixed precision for `create_supervised_trainer` and `create_supervised_evaluator` (1714, 1589)
- Updated `state.batch/state.output` lifespan in Engine (1919)

Distributed module

- Handled IterableDataset with `auto_dataloader` (2028)
- Updated Loss metric to use `required_output_keys` (2027)
- Enabled gpu support for gloo backend (2016)
- Added `safe_mode` for `idist` broadcast (1839)
- Improved `idist` to support different `init_methods` (1767)

Other improvements

- Added LR finder improvements, moved to core (2045, 1998, 1996, 1987, 1981, 1961, 1951, 1930)
- Moved param handler to core (1988)
- Added an option to store `EpochOutputStore` data on `engine.state`, moved to core (1982, 1974)
- Set seed for xla in `ignite.utils.manual_seed` (1970)
- Fixed case for Precision/Recall in `multi_label`, not averaged configuration for DDP (1646)
- Updated `PolyaxonLogger` to handle v1 and v0 (1625)
- Added Arguments `*args`, `**kwargs` to `BaseLogger.attach method` (2034)
- Enabled metric ordering on `ProgressBar` (1937)
- Updated wandb logger (1896)
- Fixed type hint for `ProgressBar` (2079)

Bug fixes

- **BC-breaking:** Improved loggers to keep configuration (1945)
- Fixed warnings in CI (2023)
- Fixed Precision for all zero predictions (2017)
- Renamed the default logger (2006)
- Fixed Accumulation metric with Nvidia/Apex (1978)
- Updated code to raise an error if SLURM is used with torch dist launcher (1976)
- Updated `nltk-smooth2` for BLEU metric (1911)
- Added full read permissions to saved file (1876) (1880)
- Fixed a bug with horovod `_do_manual_all_reduce` (1848)
- Fixed small bug in "Finetuning EfficientNet-B0 on CIFAR100" tutorial (2073)
- Fixed f-string in `mnist_save_resume_engine.py` example (2077)
- Fixed an issue when rng states accidentaly on cuda for `DeterministicEngine` (2081)

Housekeeping

<details>

<summary>
A lot of PRs
</summary>

- Test improvements (2061, 2057, 2047, 1962, 1957, 1946, 1943, 1928, 1927, 1915, 1914, 1908, 1906, 1905, 1903, 1902, 1899, 1899, 1882, 1870, 1866, 1860, 1846, 1832, 1828, 1821, 1816, 1815, 1814, 1812, 1811, 1809, 1808, 1807, 1804, 1802, 1801, 1799, 1798, 1797, 1796, 1795, 1793, 1791, 1785, 1784, 1783, 1781, 1776, 1774, 1769, 1768, 1760, 1755, 1746, 1741, 1718, 1717, 1713, 1631)
- Documentation improvements and updates (2058, 2024, 2005, 2003, 2001, 1993, 1990, 1933, 1893, 1849, 1780, 1770, 1727, 1726, 1722, 1686, 1685, 1672, 1671, 1661)
- Example improvements (1924, 1918, 1890, 1827, 1771, 1669, 1658, 1656, 1652, 1642, 1633, 1632)
- CI updates (2075, 2070, 2069, 2068, 2067, 2064, 2044, 2039, 2037, 2023, 1985, 1979, 1940, 1907, 1892, 1888, 1878, 1877, 1873, 1867, 1861, 1847, 1841, 1838, 1837, 1835, 1831, 1818, 1773, 1764, 1761, 1759, 1752, 1745, 1743, 1742, 1739, 1738, 1736, 1724, 1706, 1705, 1667, 1664, 1647)
- Code style improvements (2050, 2014, 1817, 1749, 1747, 1740, 1734, 1732, 1731, 1707, 1703)
- Added docker image test script (1733)

</details>

Acknowledgments

🎉 Thanks to our community and all our contributors for the issues, PRs and 🌟 ⭐️ 🌟 !
💯 We really appreciate your implication into the project (in alphabetical order):

01-vyom, Devanshu24, Juddd, KickItLikeShika, Moh-Yakoub, Muktan, OBITORASU, Priyansi, afzal442, ahmedo42, aksg87, aniezurawski, cozek, devrimcavusoglu, fco-dv, gucifer, log-layer, mouradmourafiq, radekosmulski, sahilg06, sdesrozis, sparkingdark, thomasjpfan, touqir14, trsvchn, vfdev-5, ydcjeff