Version 0.2 marks an important milestone in the development cycle of *atmodata*. With this release, atmodata can successfully be used as Minimum Viable Product (MVP).
:sparkles: Features
* A working end-to-end pipeline for weather forecasting on WeatherBench
* A modular system that splits pipelines into datasets (e.g. WeatherBench) and tasks (e.g. Forecasting) allowing easy expansion in the future
* An extensive datapipe builder that among others allows data sharing between worker processes, and convenient creation of dataloaders.
* Fast transfer of xarray.Dataset's via shared memory to worker processes
* Full support for all variables in WeatherBench, including efficient loading and unstacking of levels.
* 21 new IterDataPipes for xarray datastructures, torch tensors, and general purpose functions, that allow easy creation of new datasets and tasks.
:seedling: Planned
For the upcoming releases, the following features are planned:
* Full support for ERA5 at native resolution
* Horovod sharding support via a custom ReadingService
* Data normalization, including calculation of statistics and daily and hourly anomalies
* Documentation
* A minimum viable example demonstrating end-to-end training
* (Potential) performance optimization for collate()
* Profiling code specifically designed for datapipes.
* AtmoDistTask, a self-supervised training objective
* Refactoring `XrPrefetcher` to make it more general applicable