Torchmetrics

Added

- Added `average` argument to `MeanAveragePrecision` ([2018](https://github.com/Lightning-AI/torchmetrics/pull/2018))

Fixed

- Fixed bug in `PearsonCorrCoef` is updated on single samples at a time ([2019](https://github.com/Lightning-AI/torchmetrics/pull/2019))
- Fixed support for pixel-wise MSE ([2017](https://github.com/Lightning-AI/torchmetrics/pull/2017))
- Fixed bug in `MetricCollection` when used with multiple metrics that return dicts with same keys ([2027](https://github.com/Lightning-AI/torchmetrics/pull/2027))
- Fixed bug in detection intersection metrics when `class_metrics=True` resulting in wrong values ([1924](https://github.com/Lightning-AI/torchmetrics/pull/1924))
- Fixed missing attributes `higher_is_better`, `is_differentiable` for some metrics ([2028](https://github.com/Lightning-AI/torchmetrics/pull/2028))

Added

- Added source aggregated signal-to-distortion ratio (SA-SDR) metric ([1882](https://github.com/Lightning-AI/torchmetrics/pull/1882)
- Added `VisualInformationFidelity` to image package ([1830](https://github.com/Lightning-AI/torchmetrics/pull/1830))
- Added `EditDistance` to text package ([1906](https://github.com/Lightning-AI/torchmetrics/pull/1906))
- Added `top_k` argument to `RetrievalMRR` in retrieval package ([1961](https://github.com/Lightning-AI/torchmetrics/pull/1961))
- Added support for evaluating `"segm"` and `"bbox"` detection in `MeanAveragePrecision` at the same time ([1928](https://github.com/Lightning-AI/torchmetrics/pull/1928))
- Added `PerceptualPathLength` to image package ([1939](https://github.com/Lightning-AI/torchmetrics/pull/1939))
- Added support for multioutput evaluation in `MeanSquaredError` ([1937](https://github.com/Lightning-AI/torchmetrics/pull/1937))
- Added argument `extended_summary` to `MeanAveragePrecision` such that precision, recall, iou can be easily returned ([1983](https://github.com/Lightning-AI/torchmetrics/pull/1983))
- Added warning to `ClipScore` if long captions are detected and truncate ([2001](https://github.com/Lightning-AI/torchmetrics/pull/2001))
- Added `CLIPImageQualityAssessment` to multimodal package ([1931](https://github.com/Lightning-AI/torchmetrics/pull/1931))
- Added new property `metric_state` to all metrics for users to investigate currently stored tensors in memory ([2006](https://github.com/Lightning-AI/torchmetrics/pull/2006))

---

Added

- Added warning to `MeanAveragePrecision` if too many detections are observed ([1978](https://github.com/Lightning-AI/torchmetrics/pull/1978))

Fixed

- Fix support for int input for when `multidim_average="samplewise"` in classification metrics ([1977](https://github.com/Lightning-AI/torchmetrics/pull/1977))
- Fixed x/y labels when plotting confusion matrices ([1976](https://github.com/Lightning-AI/torchmetrics/pull/1976))
- Fixed IOU compute in cuda ([1982](https://github.com/Lightning-AI/torchmetrics/pull/1982))

Added

- Added warning to `PearsonCorrCoeff` if input has a very small variance for its given dtype ([1926](https://github.com/Lightning-AI/torchmetrics/pull/1926))

Changed

- Changed all non-task specific classification metrics to be true subtypes of `Metric` ([1963](https://github.com/Lightning-AI/torchmetrics/pull/1963))

Fixed

- Fixed bug in `CalibrationError` where calculations for double precision input was performed in float precision ([1919](https://github.com/Lightning-AI/torchmetrics/pull/1919))
- Fixed bug related to the `prefix/postfix` arguments in `MetricCollection` and `ClasswiseWrapper` being duplicated ([1918](https://github.com/Lightning-AI/torchmetrics/pull/1918))
- Fixed missing AUC score when plotting classification metrics that support the `score` argument ([1948](https://github.com/Lightning-AI/torchmetrics/pull/1948))

Fixed
- Fixes corner case when using `MetricCollection` together with aggregation metrics ([1896](https://github.com/Lightning-AI/torchmetrics/pull/1896))
- Fixed the use of `max_fpr` in `AUROC` metric when only one class is present ([1895](https://github.com/Lightning-AI/torchmetrics/pull/1895))
- Fixed bug related to empty predictions for `IntersectionOverUnion` metric ([1892](https://github.com/Lightning-AI/torchmetrics/pull/1892))
- Fixed bug related to `MeanMetric` and broadcasting of weights when Nans are present ([1898](https://github.com/Lightning-AI/torchmetrics/pull/1898))
- Fixed bug related to expected input format of pycoco in `MeanAveragePrecision` ([1913](https://github.com/Lightning-AI/torchmetrics/pull/1913))

Added

- Added `prefix` and `postfix` arguments to `ClasswiseWrapper` ([1866](https://github.com/Lightning-AI/torchmetrics/pull/1866))
- Added speech-to-reverberation modulation energy ratio (SRMR) metric ([1792](https://github.com/Lightning-AI/torchmetrics/pull/1792), [#1872](https://github.com/Lightning-AI/torchmetrics/pull/1872))
- Added new global arg `compute_with_cache` to control caching behaviour after `compute` method ([1754](https://github.com/Lightning-AI/torchmetrics/pull/1754))
- Added `ComplexScaleInvariantSignalNoiseRatio` for audio package ([1785](https://github.com/Lightning-AI/torchmetrics/pull/1785))
- Added `Running` wrapper for calculate running statistics ([1752](https://github.com/Lightning-AI/torchmetrics/pull/1752))
- Added`RelativeAverageSpectralError` and `RootMeanSquaredErrorUsingSlidingWindow` to image package ([816](https://github.com/Lightning-AI/torchmetrics/pull/816))
- Added support for `SpecificityAtSensitivity` Metric ([1432](https://github.com/Lightning-AI/torchmetrics/pull/1432))
- Added support for plotting of metrics through `.plot()` method (
[1328](https://github.com/Lightning-AI/torchmetrics/pull/1328),
[1481](https://github.com/Lightning-AI/torchmetrics/pull/1481),
[1480](https://github.com/Lightning-AI/torchmetrics/pull/1480),
[1490](https://github.com/Lightning-AI/torchmetrics/pull/1490),
[1581](https://github.com/Lightning-AI/torchmetrics/pull/1581),
[1585](https://github.com/Lightning-AI/torchmetrics/pull/1585),
[1593](https://github.com/Lightning-AI/torchmetrics/pull/1593),
[1600](https://github.com/Lightning-AI/torchmetrics/pull/1600),
[1605](https://github.com/Lightning-AI/torchmetrics/pull/1605),
[1610](https://github.com/Lightning-AI/torchmetrics/pull/1610),
[1609](https://github.com/Lightning-AI/torchmetrics/pull/1609),
[1621](https://github.com/Lightning-AI/torchmetrics/pull/1621),
[1624](https://github.com/Lightning-AI/torchmetrics/pull/1624),
[1623](https://github.com/Lightning-AI/torchmetrics/pull/1623),
[1638](https://github.com/Lightning-AI/torchmetrics/pull/1638),
[1631](https://github.com/Lightning-AI/torchmetrics/pull/1631),
[1650](https://github.com/Lightning-AI/torchmetrics/pull/1650),
[1639](https://github.com/Lightning-AI/torchmetrics/pull/1639),
[1660](https://github.com/Lightning-AI/torchmetrics/pull/1660),
[1682](https://github.com/Lightning-AI/torchmetrics/pull/1682),
[1786](https://github.com/Lightning-AI/torchmetrics/pull/1786),
)
- Added support for plotting of audio metrics through `.plot()` method ([1434](https://github.com/Lightning-AI/torchmetrics/pull/1434))
- Added `classes` to output from `MAP` metric ([1419](https://github.com/Lightning-AI/torchmetrics/pull/1419))
- Added Binary group fairness metrics to classification package ([1404](https://github.com/Lightning-AI/torchmetrics/pull/1404))
- Added `MinkowskiDistance` to regression package ([1362](https://github.com/Lightning-AI/torchmetrics/pull/1362))
- Added `pairwise_minkowski_distance` to pairwise package ([1362](https://github.com/Lightning-AI/torchmetrics/pull/1362))
- Added new detection metric `PanopticQuality` (
[929](https://github.com/Lightning-AI/torchmetrics/pull/929),
[1527](https://github.com/Lightning-AI/torchmetrics/pull/1527),
)
- Added `PSNRB` metric ([1421](https://github.com/Lightning-AI/torchmetrics/pull/1421))
- Added `ClassificationTask` Enum and use in metrics ([1479](https://github.com/Lightning-AI/torchmetrics/pull/1479))
- Added `ignore_index` option to `exact_match` metric ([1540](https://github.com/Lightning-AI/torchmetrics/pull/1540))
- Add parameter `top_k` to `RetrievalMAP` ([1501](https://github.com/Lightning-AI/torchmetrics/pull/1501))
- Added support for deterministic evaluation on GPU for metrics that uses `torch.cumsum` operator ([1499](https://github.com/Lightning-AI/torchmetrics/pull/1499))
- Added support for plotting of aggregation metrics through `.plot()` method ([1485](https://github.com/Lightning-AI/torchmetrics/pull/1485))
- Added support for python 3.11 ([1612](https://github.com/Lightning-AI/torchmetrics/pull/1612))
- Added support for auto clamping of input for metrics that uses the `data_range` ([1606](argument https://github.com/Lightning-AI/torchmetrics/pull/1606))
- Added `ModifiedPanopticQuality` metric to detection package ([1627](https://github.com/Lightning-AI/torchmetrics/pull/1627))
- Added `PrecisionAtFixedRecall` metric to classification package ([1683](https://github.com/Lightning-AI/torchmetrics/pull/1683))
- Added multiple metrics to detection package ([1284](https://github.com/Lightning-AI/torchmetrics/pull/1284))
* `IntersectionOverUnion`
* `GeneralizedIntersectionOverUnion`
* `CompleteIntersectionOverUnion`
* `DistanceIntersectionOverUnion`
- Added `MultitaskWrapper` to wrapper package ([1762](https://github.com/Lightning-AI/torchmetrics/pull/1762))
- Added `RelativeSquaredError` metric to regression package ([1765](https://github.com/Lightning-AI/torchmetrics/pull/1765))
- Added `MemorizationInformedFrechetInceptionDistance` metric to image package ([1580](https://github.com/Lightning-AI/torchmetrics/pull/1580))


Changed

- Changed `permutation_invariant_training` to allow using a `'permutation-wise'` metric function ([1794](https://github.com/Lightning-AI/torchmetrics/pull/1794))
- Changed `update_count` and `update_called` from private to public methods ([1370](https://github.com/Lightning-AI/torchmetrics/pull/1370))
- Raise exception for invalid kwargs in Metric base class ([1427](https://github.com/Lightning-AI/torchmetrics/pull/1427))
- Extend `EnumStr` raising `ValueError` for invalid value ([1479](https://github.com/Lightning-AI/torchmetrics/pull/1479))
- Improve speed and memory consumption of binned `PrecisionRecallCurve` with large number of samples ([1493](https://github.com/Lightning-AI/torchmetrics/pull/1493))
- Changed `__iter__` method from raising `NotImplementedError` to `TypeError` by setting to `None` ([1538](https://github.com/Lightning-AI/torchmetrics/pull/1538))
- `FID` metric will now raise an error if too few samples are provided ([1655](https://github.com/Lightning-AI/torchmetrics/pull/1655))
- Allowed FID with `torch.float64` ([1628](https://github.com/Lightning-AI/torchmetrics/pull/1628))
- Changed `LPIPS` implementation to no more rely on third-party package ([1575](https://github.com/Lightning-AI/torchmetrics/pull/1575))
- Changed FID matrix square root calculation from `scipy` to `torch` ([1708](https://github.com/Lightning-AI/torchmetrics/pull/1708))
- Changed calculation in `PearsonCorrCoeff` to be more robust in certain cases ([1729](https://github.com/Lightning-AI/torchmetrics/pull/1729))
- Changed `MeanAveragePrecision` to `pycocotools` backend ([1832](https://github.com/Lightning-AI/torchmetrics/pull/1832))


Deprecated

- Deprecated domain metrics import from package root (
[1685](https://github.com/Lightning-AI/torchmetrics/pull/1685),
[1694](https://github.com/Lightning-AI/torchmetrics/pull/1694),
[1696](https://github.com/Lightning-AI/torchmetrics/pull/1696),
[1699](https://github.com/Lightning-AI/torchmetrics/pull/1699),
[1703](https://github.com/Lightning-AI/torchmetrics/pull/1703),
)


Removed

- Support for python 3.7 ([1640](https://github.com/Lightning-AI/torchmetrics/pull/1640))


Fixed

- Fixed support in `MetricTracker` for `MultioutputWrapper` and nested structures ([1608](https://github.com/Lightning-AI/torchmetrics/pull/1608))
- Fixed restrictive check in `PearsonCorrCoef` ([1649](https://github.com/Lightning-AI/torchmetrics/pull/1649))
- Fixed integration with `jsonargparse` and `LightningCLI` ([1651](https://github.com/Lightning-AI/torchmetrics/pull/1651))
- Fixed corner case in calibration error for zero confidence input ([1648](https://github.com/Lightning-AI/torchmetrics/pull/1648))
- Fix precision-recall curve based computations for float target ([1642](https://github.com/Lightning-AI/torchmetrics/pull/1642))
- Fixed missing kwarg squeeze in `MultiOutputWrapper` ([1675](https://github.com/Lightning-AI/torchmetrics/pull/1675))
- Fixed padding removal for 3d input in `MSSSIM` ([1674](https://github.com/Lightning-AI/torchmetrics/pull/1674))
- Fixed `max_det_threshold` in MAP detection ([1712](https://github.com/Lightning-AI/torchmetrics/pull/1712))
- Fixed states being saved in metrics that use `register_buffer` ([1728](https://github.com/Lightning-AI/torchmetrics/pull/1728))
- Fixed states not being correctly synced and device transferred in `MeanAveragePrecision` for `iou_type="segm"` ([1763](https://github.com/Lightning-AI/torchmetrics/pull/1763))
- Fixed use of `prefix` and `postfix` in nested `MetricCollection` ([1773](https://github.com/Lightning-AI/torchmetrics/pull/1773))
- Fixed `ax` plotting logging in `MetricCollection ([1783](https://github.com/Lightning-AI/torchmetrics/pull/1783))
- Fixed lookup for punkt sources being downloaded in `RougeScore` ([1789](https://github.com/Lightning-AI/torchmetrics/pull/1789))
- Fixed integration with lightning for `CompositionalMetric` ([1761](https://github.com/Lightning-AI/torchmetrics/pull/1761))
- Fixed several bugs in `SpectralDistortionIndex` metric ([1808](https://github.com/Lightning-AI/torchmetrics/pull/1808))
- Fixed bug for corner cases in `MatthewsCorrCoef` (
[1812](https://github.com/Lightning-AI/torchmetrics/pull/1812),
[1863](https://github.com/Lightning-AI/torchmetrics/pull/1863)
)
- Fixed support for half precision in `PearsonCorrCoef` ([1819](https://github.com/Lightning-AI/torchmetrics/pull/1819))
- Fixed number of bugs related to `average="macro"` in classification metrics ([1821](https://github.com/Lightning-AI/torchmetrics/pull/1821))
- Fixed off-by-one issue when `ignore_index = num_classes + 1` in Multiclass-jaccard ([1860](https://github.com/Lightning-AI/torchmetrics/pull/1860))

---