Pytorch-metric-learning

Bug fixes
- 420
- 422

New features

[CentroidTripletLoss](https://kevinmusgrave.github.io/pytorch-metric-learning/losses/#centroidtripletloss)
Implementation of [On the Unreasonable Effectiveness of Centroids in Image Retrieval](https://arxiv.org/pdf/2104.13643.pdf)

[VICRegLoss](https://kevinmusgrave.github.io/pytorch-metric-learning/losses/#vicregloss)
Implementation of [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://arxiv.org/pdf/2105.04906.pdf)

[AccuracyCalculator](https://kevinmusgrave.github.io/pytorch-metric-learning/accuracy_calculation/)
- Added mean reciprocal rank as an accuracy metric. Available as "mean_reciprocal_rank".
- Added return_per_class argument for AccuracyCalculator. This is like avg_of_avgs but returns the accuracy per class, instead of averaging them for you.


Related issues
369
372
374
394

Contributors
Thanks to cwkeam and mlw214!

Reference embeddings for tuple losses
You can separate the source of anchors and positive/negatives. In the example below, anchors will be selected from embeddings and positives/negatives will be selected from ref_emb.

python
loss_fn = TripletMarginLoss()
loss = loss_fn(embeddings, labels, ref_emb=ref_emb, ref_labels=ref_labels)


Efficient mode for DistributedLossWrapper
- efficient=True: each process uses its own embeddings for anchors, and the gathered embeddings for positives/negatives. Gradients will **not** be equal to those in non-distributed code, but the benefit is reduced memory and faster training.
- efficient=False: each process uses gathered embeddings for both anchors and positives/negatives. Gradients will be equal to those in non-distributed code, but at the cost of doing unnecessary operations (i.e. doing computations where both anchors and positives/negatives have no gradient).

The default is False. You can set it to True like this:

python
from pytorch_metric_learning import losses
from pytorch_metric_learning.utils import distributed as pml_dist

loss_func = losses.ContrastiveLoss()
loss_func = pml_dist.DistributedLossWrapper(loss_func, efficient=True)

Documentation: https://kevinmusgrave.github.io/pytorch-metric-learning/distributed/

Customizing k-nearest-neighbors for AccuracyCalculator
You can use a different type of faiss index:
python
import faiss
from pytorch_metric_learning.utils.accuracy_calculator import AccuracyCalculator
from pytorch_metric_learning.utils.inference import FaissKNN

knn_func = FaissKNN(index_init_fn=faiss.IndexFlatIP, gpus=[0,1,2])
ac = AccuracyCalculator(knn_func=knn_func)


You can also use a custom distance function:
python
from pytorch_metric_learning.distances import SNRDistance
from pytorch_metric_learning.utils.inference import CustomKNN

knn_func = CustomKNN(SNRDistance())
ac = AccuracyCalculator(knn_func=knn_func)


Relevant docs:
- [Accuracy Calculation](https://kevinmusgrave.github.io/pytorch-metric-learning/accuracy_calculation/)
- [FaissKNN](https://kevinmusgrave.github.io/pytorch-metric-learning/inference_models/#faissknn)
- [CustomKNN](https://kevinmusgrave.github.io/pytorch-metric-learning/inference_models/#customknn)


Issues resolved
https://github.com/KevinMusgrave/pytorch-metric-learning/issues/204
https://github.com/KevinMusgrave/pytorch-metric-learning/issues/251
https://github.com/KevinMusgrave/pytorch-metric-learning/issues/256
https://github.com/KevinMusgrave/pytorch-metric-learning/issues/292
https://github.com/KevinMusgrave/pytorch-metric-learning/issues/330
https://github.com/KevinMusgrave/pytorch-metric-learning/issues/337
https://github.com/KevinMusgrave/pytorch-metric-learning/issues/345
https://github.com/KevinMusgrave/pytorch-metric-learning/issues/347
https://github.com/KevinMusgrave/pytorch-metric-learning/issues/349
https://github.com/KevinMusgrave/pytorch-metric-learning/issues/353
https://github.com/KevinMusgrave/pytorch-metric-learning/issues/359
https://github.com/KevinMusgrave/pytorch-metric-learning/issues/361
https://github.com/KevinMusgrave/pytorch-metric-learning/issues/362
https://github.com/KevinMusgrave/pytorch-metric-learning/issues/363
https://github.com/KevinMusgrave/pytorch-metric-learning/issues/368
https://github.com/KevinMusgrave/pytorch-metric-learning/issues/376
https://github.com/KevinMusgrave/pytorch-metric-learning/issues/380

Contributors
Thanks to yutanakamura-tky and KinglittleQ for pull requests, and mensaochun for providing helpful code in 380

Bug fixes

- Accuracy Calculation bug in GlobalTwoStreamEmbeddingSpaceTester (301)
- Mixed precision bug in convert_to_weights (300)

Features

- [HierarchicalSampler](https://kevinmusgrave.github.io/pytorch-metric-learning/samplers/#hierarchicalsampler)
- Improved functionality for [InferenceModel](https://kevinmusgrave.github.io/pytorch-metric-learning/inference_models/#inferencemodel) (296 and 304)
- train_indexer now accepts a dataset
- also added functions save_index, load_index, and add_to_indexer
- Added power argument to LpRegularizer (299)
- Return exception if labels has more than 1 dimension (307)
- [Added a global flag](https://kevinmusgrave.github.io/pytorch-metric-learning/common_functions/#collect_stats) for turning on/off collect_stats (311)
- TripletMarginLoss smooth variant uses the input margin now (315)
- [Use package-specific logger, "PML"](https://kevinmusgrave.github.io/pytorch-metric-learning/common_functions/#logger), instead of root logger (318)
- Cleaner key verification in the trainers (102)

Thanks to elias-ramzi, gkouros, vltanh, and Hummer12007

AccuracyCalculator breaking change (issue 290)
The k parameter in AccuracyCalculator has a new behavior. The allowed values are:

* None. This means k will be set to the total number of reference embeddings.
* An integer greater than 0. This means k will be set to the input integer.
* "max_bin_count". This means k will be set to max(bincount(reference_labels)) - self_count where self_count == 1 if the query and reference embeddings come from the same source.

The old behavior is described [here](https://kevinmusgrave.github.io/pytorch-metric-learning/accuracy_calculation/#warning-for-versions-0997).

If your dataset is large, you might find the k-nn search is now very slow. This is because the new default behavior is to set k to len(reference_embeddings). To avoid this, you can set k to a number, like k = 1000 or try k = "max_bin_count" to get behavior similar (though not identical) to the old default.

Apologies for the drastic change. I'm hoping to have things stable and following semantic versioning when v1.0 arrives.


Bug fixes
* lmu.convert_to_triplets has been fixed (291)
* Losses and miners should now be compatible with autocast (293)

New features / improvements
* The loss used in [Supervised Contrastive Learning](https://arxiv.org/abs/2004.11362). Documentation: [SupConLoss](https://kevinmusgrave.github.io/pytorch-metric-learning/losses/#supconloss). By fjsj (281, 288)
* Vectorized convert_to_triplets (279)

Bug fixes
- Small fix for NTXentLoss with no negative pairs 272
- Fixed .detach() bug in NTXentLoss 282
- Fixed parameter override bug in MatchFinder.get_matching_pairs() 286 by joaqo

New features and improvements

AccuracyCalculator now uses torch instead of numpy
- All the calculations (except for NMI and AMI) are done with torch. Calculations will be done on the same device and dtype as the input query tensor.
- You can still pass numpy arrays into AccuracyCalculator.get_accuracy, but the arrays will be immediately converted to torch tensors.

Faster custom label comparisons in AccuracyCalculator
- See 264 by mlopezantequera

Numerical stability improvement for DistanceWeightedMiner
See 278 by z1w

UniformHistogramMiner
This is like DistanceWeightedMiner, except that it works well with high dimension embeddings, and works with any distance metric (not just L2 normalized distance). [Documentation](https://kevinmusgrave.github.io/pytorch-metric-learning/miners/#uniformhistogramminer)

PerAnchorReducer
This converts unreduced pairs to unreduced elements. For example, NTXentLoss returns losses per positive pair. If you used PerAnchorReducer with NTXentLoss, then the losses per pair would first be converted to losses per batch element, before being passed to the inner reducer. See the [documentation](https://kevinmusgrave.github.io/pytorch-metric-learning/reducers/#peranchorreducer)

BaseTester no longer converts embeddings from torch to numpy
This includes the get_all_embeddings function. If you want get_all_embeddings to return numpy arrays, you can set the return_as_numpy flag to True:
python
embeddings, labels = tester.get_all_embeddings(dataset, model, return_as_numpy=True)

The embeddings are converted to numpy only for the visualizer and visualizer_hook, if specified.

Reduced usage of .to(device) and .type(dtype)
Tensors are initialized on device and with the necessary dtype, and they are moved to device and cast to dtypes only when necessary. See [this code snippet](https://github.com/KevinMusgrave/pytorch-metric-learning/blob/3c354998a218409d3ac6f00a9662696acf97ff96/src/pytorch_metric_learning/utils/common_functions.py#L460-L473) for details.

Simplified DivisorReducer
Replaced "divisor_summands" with "divisor".