Lightly

Improved Video Loading

Loading frames from videos was updated and we now verify that the timestamp of the loaded frame is correct. This fixes a bug where sometimes a frame with a different timestamp than the one requested by the user was loaded. Lightly will now also warn you if the correct frame could not be found.

Documentation Updates
We added many documentation updates on how you can use and interact with the Lightly Platform.

Models
- [Bootstrap your own latent: A new approach to self-supervised Learning, 2020](https://arxiv.org/abs/2006.07733)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021](https://arxiv.org/abs/2103.03230)
- [DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021](https://arxiv.org/abs/2104.14294)
- [SimSiam: Exploring Simple Siamese Representation Learning, 2020](https://arxiv.org/abs/2011.10566)
- [MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019](https://arxiv.org/abs/1911.05722)
- [SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020](https://arxiv.org/abs/2002.05709)
- [NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021](https://arxiv.org/pdf/2104.14548.pdf)
- [SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020](https://arxiv.org/abs/2006.09882)

SimSiam and bug fixes

SimSiam fixes
gergopool noticed and fixed two minor issues with the `SimSiam` implementation! Thanks a lot 🙂

Bug fix: Embedding doesn't work with prefetch generator
Fixed the bug introduced with 748 that the embedding images with `lightly` didn't work when the `prefetch_generator` package is installed.

Models
- [Bootstrap your own latent: A new approach to self-supervised Learning, 2020](https://arxiv.org/abs/2006.07733)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021](https://arxiv.org/abs/2103.03230)
- [DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021](https://arxiv.org/abs/2104.14294)
- [SimSiam: Exploring Simple Siamese Representation Learning, 2020](https://arxiv.org/abs/2011.10566)
- [MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019](https://arxiv.org/abs/1911.05722)
- [SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020](https://arxiv.org/abs/2002.05709)
- [NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021](https://arxiv.org/pdf/2104.14548.pdf)
- [SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020](https://arxiv.org/abs/2006.09882)

Improved embedding progress bar, Export to Label, Docs improvements
Improved embedding progress bar
The embedding progress bar now shows the number of embedded images instead of embedded batches.
Label export
You can now use the `ApiWorkflowClient` to export data from the LightlyPlatform as a .json file to be imported into a label tool.
Docs improvements
Some errors in the docs have been fixed.

Models
- [Bootstrap your own latent: A new approach to self-supervised Learning, 2020](https://arxiv.org/abs/2006.07733)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021](https://arxiv.org/abs/2103.03230)
- [DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021](https://arxiv.org/abs/2104.14294)
- [SimSiam: Exploring Simple Siamese Representation Learning, 2020](https://arxiv.org/abs/2011.10566)
- [MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019](https://arxiv.org/abs/1911.05722)
- [SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020](https://arxiv.org/abs/2002.05709)
- [NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021](https://arxiv.org/pdf/2104.14548.pdf)
- [SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020](https://arxiv.org/abs/2006.09882)

DINO tricks, Imagenette benchmarks, API improvements

DINO tricks
DINOHead now allows to freeze the last layer which stabilizes the model performance.
DINOHead now also allows to normalise the last layer.
This was implemented by Atharva-Phatak. Thank you very much!

Imagenette benchmarks
We now include benchmarks of all models on Imagenette.

API Improvements
Better documentation of custom metadata
The CLI command to upload custom metadata is now included in the command line tool examples.
Better dataset upsizing
Upsizing a dataset in the Lightly Platform by adding more samples to it now cannot happen accidentally anymore, instead you have to specify `append=True`. Furthermore, bugs regarding appending new custom metadata have been fixed.
Create ApiWorkflowClient with token from env
When creating an `ApiWorkflowClient`, you can now pass the token as environment variable `LIGHTLY_TOKEN` instead of as argument.
Bugfixes in `check_embeddings()`
When checking embedding files, now columns like `masked` and `selected` are accounted for properly.

Models
- [Bootstrap your own latent: A new approach to self-supervised Learning, 2020](https://arxiv.org/abs/2006.07733)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021](https://arxiv.org/abs/2103.03230)
- [DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021](https://arxiv.org/abs/2104.14294)
- [SimSiam: Exploring Simple Siamese Representation Learning, 2020](https://arxiv.org/abs/2011.10566)
- [MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019](https://arxiv.org/abs/1911.05722)
- [SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020](https://arxiv.org/abs/2002.05709)
- [NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021](https://arxiv.org/pdf/2104.14548.pdf)
- [SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020](https://arxiv.org/abs/2006.09882)

Active Learning for Keypoint Detection and New API Workflows

Active Learning: Keypoint Detection
707 added an active learning scorer for keypoint detection. This means that you can now do active learning for keypoint detection with Lightly!

New API Workflows
With 712 and 714 we added all the necessities to [configure and run the compute worker](https://docs.lightly.ai/docker/integration/docker_with_datasource.html) from your Python code.

Access Shared Datasets
722 makes it possible to access datasets which were shared with you in the Lightly web-app from your Python code as if they were your own.

Documentation
We made many smaller changes and fixes in the docs 🙂

Models
- [Bootstrap your own latent: A new approach to self-supervised Learning, 2020](https://arxiv.org/abs/2006.07733)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021](https://arxiv.org/abs/2103.03230)
- [DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021](https://arxiv.org/abs/2104.14294)
- [SimSiam: Exploring Simple Siamese Representation Learning, 2020](https://arxiv.org/abs/2011.10566)
- [MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019](https://arxiv.org/abs/1911.05722)
- [SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020](https://arxiv.org/abs/2002.05709)
- [NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021](https://arxiv.org/pdf/2104.14548.pdf)
- [SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020](https://arxiv.org/abs/2006.09882)

Bugfixes and Documentation Improvements

Bugfix in train_model_and_embed_images()
Fixed the bug that the `train_model_and_embed_images()` function did not return the embeddings, filenames and labels. Thanks for finding this bug opassos!

Bugfix in video loader
When using the video_loader backend of torchvision, it could fail when training for multiple epochs and had a linearly growing demand for memory and file handlers. This was fixed.

Documentation update in Readme.md
The benchmark table for CIFAR10 was fixed and the DINO model added to the list of supported models.

Models
- [Bootstrap your own latent: A new approach to self-supervised Learning, 2020](https://arxiv.org/abs/2006.07733)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021](https://arxiv.org/abs/2103.03230)
- [DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021](https://arxiv.org/abs/2104.14294)
- [SimSiam: Exploring Simple Siamese Representation Learning, 2020](https://arxiv.org/abs/2011.10566)
- [MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019](https://arxiv.org/abs/1911.05722)
- [SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020](https://arxiv.org/abs/2002.05709)
- [NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021](https://arxiv.org/pdf/2104.14548.pdf)
- [SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020](https://arxiv.org/abs/2006.09882)