Pytorch-metric-learning

New Features

Thanks to mlopezantequera for adding the following features!

Testers: allow any combination of query and reference sets (250)
To evaluate different combinations of query and reference sets, use the splits_to_eval argument for tester.test().

For example, let's say your dataset_dict has two keys: "dataset_a" and "train".

- The default splits_to_eval = None is equivalent to:
python
splits_to_eval = [('dataset_a', ['dataset_a']), ('train', ['train'])]

- dataset_a as the query, and train as the reference:
python
splits_to_eval = [('dataset_a', ['train'])]

- dataset_a as the query, and dataset_a + train as the reference:
python
splits_to_eval = [('dataset_a', ['dataset_a', 'train'])]


Then pass splits_to_eval to tester.test:
python
tester.test(dataset_dict, epoch, model, splits_to_eval = splits_to_eval)


Note that this new feature makes the old reference_set init argument obsolete, so reference_set has been removed.



AccuracyCalculator: allow arbitrary label comparion functions (254)
AccuracyCalculator now has an optional init argument, label_comparison_fn, which is a function that compares two numpy arrays of labels and returns a boolean array. The default is numpy.equal. If a custom function is used, then you must exclude clustering based metrics ("NMI" and "AMI"). The following is an example of a custom function for two-dimensional labels. It returns True if the 0th column matches, and the 1st column does **not** match:
python
def example_label_comparison_fn(x, y):
return (x[:, 0] == y[:, 0]) & (x[:, 1] != y[:, 1])

AccuracyCalculator(exclude=("NMI", "AMI"),
label_comparison_fn=example_label_comparison_fn)


Other Changes

- BaseTrainer and BaseTester now take in an optional dtype argument. This is the type that the dataset output will be converted to, e.g. torch.float16. If set to the default value of None, then no type casting will be done.
- Removed self.dim_reduced_embeddings from BaseTester and the associated code in HookContainer, due to lack of use.
- tester.test() now returns all_accuracies, whereas before, it returned nothing and you'd have to access all_accuracies either through the end_of_testing_hook or by accessing tester.all_accuracies.
- tester.embeddings_and_labels is deleted at the end of tester.test() to free up memory.

New

BatchEasyHardMiner
This new miner is an implementation of [Improved Embeddings with Easy Positive Triplet Mining](https://openaccess.thecvf.com/content_WACV_2020/papers/Xuan_Improved_Embeddings_with_Easy_Positive_Triplet_Mining_WACV_2020_paper.pdf). See [the documentation](https://kevinmusgrave.github.io/pytorch-metric-learning/miners/#batcheasyhardminer). Thanks marijnl!

New metric added to AccuracyCalculator
The new metric is mean_average_precision, which is the commonly used k-nn based mAP in information retrieval.
Note that this differs from the already existing metric, mean_average_precision_at_r.

Bug fixes

- dtype casting in MultiSimilarityMiner changed to work with autocast. See 233 by thinline72
- Added logic for dealing with zero rows in the weight matrix in DistanceWeightedMiner by ignoring them. For example, if the entire weight matrix is 0, then no triplets will be returned. Previously, the zero rows would cause a RuntimeError. See 230 by tpanum

Various bug fixes and improvements

- A list or dictionary of miners can be passed into MultipleLosses. 212
- Fixed bug where MultipleLosses failed in list mode. 213
- Fixed bug where IntraPairVarianceLoss and MarginLoss were overriding sub_loss_names instead of _sub_loss_names. This likely caused embedding regularizers to have no effect for these two losses. 215
- ModuleWithRecordsAndReducer now creates copies of the input reducer when necessary. 216
- Moved cos.clone() inside torch.no_grad() in RegularFaceRegularizer. Should be more efficient? 219
- In utils.inference, moved faiss import inside of FaissIndexer since that is the only class that requires it. 222
- Added a copy_weights init argument to LogitGetter, to make copying optional 223

Small update

- Optimized get_random_triplet_indices, so if you were using DistanceWeightedMiner, or if you ever set the triplets_per_anchor argument to something other than "all" anywhere in your code, it should run a lot faster now. Thanks AlexSchuy

New Features

DistributedLossWrapper and DistributedMinerWrapper
Added [DistributedLossWrapper and DistributedMinerWrapper](https://kevinmusgrave.github.io/pytorch-metric-learning/distributed). Wrap a loss or miner with these when using PyTorch's DistributedDataParallel (i.e. multiprocessing). Most of the code is by JohnGiorgi (https://github.com/JohnGiorgi/DeCLUTR).

python
from pytorch_metric_learning import losses, miners
from pytorch_metric_learning.utils import distributed as pml_dist
loss_func = pml_dist.DistributedLossWrapper(loss = losses.ContrastiveLoss())
miner = pml_dist.DistributedMinerWrapper(miner = miners.MultiSimilarityMiner())


For a working example, see the ["Multiprocessing with DistributedDataParallel"](https://github.com/KevinMusgrave/pytorch-metric-learning/tree/master/examples#simple-examples) notebook.

Added enqueue_idx to CrossBatchMemory
Now you can make CrossBatchMemory work with [MoCo](https://arxiv.org/pdf/1911.05722.pdf). This adds a great deal of flexibility to the MoCo framework, because you can use any tuple loss and tuple miner in CrossBatchMemory.

Previously this wasn't possible because all embeddings passed into CrossBatchMemory would go into the memory queue. In contrast, MoCo only queues the momentum encoder's embeddings.

The new enqueue_idx argument lets you do this, by specifying which embeddings should be added to memory. Here's a modified snippet from the [MoCo on CIFAR10](https://github.com/KevinMusgrave/pytorch-metric-learning/tree/master/examples#simple-examples) notebook:
python
from pytorch_metric_learning.losses import CrossBatchMemory, NTXentLoss

loss_fn = CrossBatchMemory(loss = NTXentLoss(), embedding_size = 64, memory_size = 16384)

snippet from the training loop
for images, _ in train_loader:
...
previous_max_label = torch.max(loss_fn.label_memory)
num_pos_pairs = encQ_out.size(0)
labels = torch.arange(0, num_pos_pairs)
labels = torch.cat((labels , labels)).to(device)

add an offset so that the labels do not overlap with any labels in the memory queue
labels += previous_max_label + 1

we want to enqueue the output of encK, which is the 2nd half of the batch
enqueue_idx = torch.arange(num_pos_pairs, num_pos_pairs*2)

all_enc = torch.cat([encQ_out, encK_out], dim=0)

now only encK_out will be added to the memory queue
loss = loss_fn(all_enc, labels, enqueue_idx = enqueue_idx)
...


Check out the [MoCo on CIFAR10](https://github.com/KevinMusgrave/pytorch-metric-learning/tree/master/examples#simple-examples) notebook to see the entire script.


TuplesToWeightsSampler
This is a [simple offline miner](https://kevinmusgrave.github.io/pytorch-metric-learning/samplers/#tuplestoweightssampler). It does the following:
1. Take a random subset of your dataset, if you provide subset_size
2. Use a specified miner to mine tuples from the subset dataset.
3. Compute weights based on how often an element appears in the mined tuples.
4. Randomly sample, using the weights as probabilities.

python
from pytorch_metric_learning.samplers import TuplesToWeightsSampler
from pytorch_metric_learning.miners import MultiSimilarityMiner

miner = MultiSimilarityMiner(epsilon=-0.2)
sampler = TuplesToWeightsSampler(model, miner, dataset, subset_size = 5000)
then pass the sampler into your Dataloader


LogitGetter
Added [utils.inference.LogitGetter](https://kevinmusgrave.github.io/pytorch-metric-learning/inference_models/#logitgetter) to make it easier to compute logits of classifier loss functions.
python
from pytorch_metric_learning.losses import ArcFaceLoss
from pytorch_metric_learning.utils.inference import LogitGetter

loss_fn = ArcFaceLoss(num_classes = 100, embedding_size = 512)
LG = LogitGetter(loss_fn)
logits = LG(embeddings)


Other
- Added optional batch_size argument to MPerClassSampler. If you pass in this argument, then each batch is guaranteed to have m samples per class. Otherwise, most batches will have m samples per class, but it's not guaranteed for every batch. Note there restrictions on the values of m and batch_size. For example, batch_size must be a multiple of m. For all the restrictions, see [the documentation](https://kevinmusgrave.github.io/pytorch-metric-learning/samplers/#mperclasssampler).

- Added trainable_attributes to BaseTrainer and to standardize the set_to_train and set_to_eval functions.

- Added save_models init argument to HookContainer. If set to False then models will not be saved.

- Added losses_sizes as a stat for BaseReducer

- Added a type check and conversion in common_functions.labels_to_indices to go from torch tensor to numpy

Bug Fixes and Improvements
- Fixed CircleLoss bug, by improving the logsumexp keep_mask implementation. See https://github.com/KevinMusgrave/pytorch-metric-learning/issues/173
- Fixed convert_to_weights bug, which caused a runtime error when an empty indices_tuple was passed in. See https://github.com/KevinMusgrave/pytorch-metric-learning/issues/174
- ProxyAnchorLoss now adds miner weights to the exponents which are fed to logsumexp. This is equivalent to scaling each loss component by e^(miner_weight). The previous behavior was to scale each loss component by just miner_weight.

Other updates
- Added an [example notebook](https://github.com/KevinMusgrave/pytorch-metric-learning/tree/master/examples#simple-examples) which shows how to use a customized loss + miner in a simple training loop.
- A new [arxiv paper](https://arxiv.org/abs/2008.09164) and an improved Readme give a better high level explanation of the library.
- Added better explanations for the [testers](https://kevinmusgrave.github.io/pytorch-metric-learning/testers/) and the [default accuracy metrics](https://kevinmusgrave.github.io/pytorch-metric-learning/accuracy_calculation/#explanations-of-the-default-accuracy-metrics)