Lightly

Decoupled Contrastive Learning
We have implemented the Decoupled Contrastive Learning (DCL) loss. It allows faster training with smaller batch sizes than other self-supervised learning models.

Documentation: https://docs.lightly.ai/examples/dcl.html
Decoupled Contrastive Learning paper: https://arxiv.org/abs/2110.06848

Model heads with default parameters
All model heads have now default parameters following the values of the original papers.

Create custom metadata config
Custom metadata in the Lightly Platform can now easily be configured via `ApiWorkflowClient.create_custom_metadata_config()`

Progress bar for video dataset initialization
Constructing a `LightlyDataset` with large video datasets can take long, as all frames in all videos have to be counted. We added a tqdm progress bar for it.

Models
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021](https://arxiv.org/abs/2103.03230)
- [Bootstrap your own latent: A new approach to self-supervised Learning, 2020](https://arxiv.org/abs/2006.07733)
- [DCL: Decoupled Contrastive Learning, 2021](https://arxiv.org/abs/2110.06848)
- [DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021](https://arxiv.org/abs/2104.14294)
- [MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019](https://arxiv.org/abs/1911.05722)
- [NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021](https://arxiv.org/pdf/2104.14548.pdf)
- [SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020](https://arxiv.org/abs/2002.05709)
- [SimSiam: Exploring Simple Siamese Representation Learning, 2020](https://arxiv.org/abs/2011.10566)
- [SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020](https://arxiv.org/abs/2006.09882)

More stable image download
Lightly now also retries an image download if the read url exists at first, but becomes unavailable during the download.

More stable video frame download
Some videos drop frames if seeking to keyframes is used. Lightly now detects this case and handles it by retrying the reading without seeking.

Models
- [Bootstrap your own latent: A new approach to self-supervised Learning, 2020](https://arxiv.org/abs/2006.07733)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021](https://arxiv.org/abs/2103.03230)
- [DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021](https://arxiv.org/abs/2104.14294)
- [SimSiam: Exploring Simple Siamese Representation Learning, 2020](https://arxiv.org/abs/2011.10566)
- [MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019](https://arxiv.org/abs/1911.05722)
- [SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020](https://arxiv.org/abs/2002.05709)
- [NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021](https://arxiv.org/pdf/2104.14548.pdf)
- [SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020](https://arxiv.org/abs/2006.09882)

Improved Video Loading

Loading frames from videos was updated and we now verify that the timestamp of the loaded frame is correct. This fixes a bug where sometimes a frame with a different timestamp than the one requested by the user was loaded. Lightly will now also warn you if the correct frame could not be found.

Documentation Updates
We added many documentation updates on how you can use and interact with the Lightly Platform.

Models
- [Bootstrap your own latent: A new approach to self-supervised Learning, 2020](https://arxiv.org/abs/2006.07733)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021](https://arxiv.org/abs/2103.03230)
- [DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021](https://arxiv.org/abs/2104.14294)
- [SimSiam: Exploring Simple Siamese Representation Learning, 2020](https://arxiv.org/abs/2011.10566)
- [MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019](https://arxiv.org/abs/1911.05722)
- [SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020](https://arxiv.org/abs/2002.05709)
- [NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021](https://arxiv.org/pdf/2104.14548.pdf)
- [SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020](https://arxiv.org/abs/2006.09882)

SimSiam and bug fixes

SimSiam fixes
gergopool noticed and fixed two minor issues with the `SimSiam` implementation! Thanks a lot 🙂

Bug fix: Embedding doesn't work with prefetch generator
Fixed the bug introduced with 748 that the embedding images with `lightly` didn't work when the `prefetch_generator` package is installed.

Models
- [Bootstrap your own latent: A new approach to self-supervised Learning, 2020](https://arxiv.org/abs/2006.07733)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021](https://arxiv.org/abs/2103.03230)
- [DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021](https://arxiv.org/abs/2104.14294)
- [SimSiam: Exploring Simple Siamese Representation Learning, 2020](https://arxiv.org/abs/2011.10566)
- [MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019](https://arxiv.org/abs/1911.05722)
- [SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020](https://arxiv.org/abs/2002.05709)
- [NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021](https://arxiv.org/pdf/2104.14548.pdf)
- [SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020](https://arxiv.org/abs/2006.09882)

Improved embedding progress bar, Export to Label, Docs improvements
Improved embedding progress bar
The embedding progress bar now shows the number of embedded images instead of embedded batches.
Label export
You can now use the `ApiWorkflowClient` to export data from the LightlyPlatform as a .json file to be imported into a label tool.
Docs improvements
Some errors in the docs have been fixed.

Models
- [Bootstrap your own latent: A new approach to self-supervised Learning, 2020](https://arxiv.org/abs/2006.07733)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021](https://arxiv.org/abs/2103.03230)
- [DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021](https://arxiv.org/abs/2104.14294)
- [SimSiam: Exploring Simple Siamese Representation Learning, 2020](https://arxiv.org/abs/2011.10566)
- [MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019](https://arxiv.org/abs/1911.05722)
- [SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020](https://arxiv.org/abs/2002.05709)
- [NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021](https://arxiv.org/pdf/2104.14548.pdf)
- [SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020](https://arxiv.org/abs/2006.09882)

DINO tricks, Imagenette benchmarks, API improvements

DINO tricks
DINOHead now allows to freeze the last layer which stabilizes the model performance.
DINOHead now also allows to normalise the last layer.
This was implemented by Atharva-Phatak. Thank you very much!

Imagenette benchmarks
We now include benchmarks of all models on Imagenette.

API Improvements
Better documentation of custom metadata
The CLI command to upload custom metadata is now included in the command line tool examples.
Better dataset upsizing
Upsizing a dataset in the Lightly Platform by adding more samples to it now cannot happen accidentally anymore, instead you have to specify `append=True`. Furthermore, bugs regarding appending new custom metadata have been fixed.
Create ApiWorkflowClient with token from env
When creating an `ApiWorkflowClient`, you can now pass the token as environment variable `LIGHTLY_TOKEN` instead of as argument.
Bugfixes in `check_embeddings()`
When checking embedding files, now columns like `masked` and `selected` are accounted for properly.

Models
- [Bootstrap your own latent: A new approach to self-supervised Learning, 2020](https://arxiv.org/abs/2006.07733)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021](https://arxiv.org/abs/2103.03230)
- [DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021](https://arxiv.org/abs/2104.14294)
- [SimSiam: Exploring Simple Siamese Representation Learning, 2020](https://arxiv.org/abs/2011.10566)
- [MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019](https://arxiv.org/abs/1911.05722)
- [SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020](https://arxiv.org/abs/2002.05709)
- [NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021](https://arxiv.org/pdf/2104.14548.pdf)
- [SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020](https://arxiv.org/abs/2006.09882)