Adios2

Summary
This is the fourth pre-release of the ADaptable Input Output System, ADIOS, version 2 (v2.2.0). The primary focus of this beta release is increased performance through I/O aggregation. This version is not intended for production quality purposes.

Features
Major additions from the previous v2.1.1 Beta release include process aggregation and multi-block I/O.

Aggregation
So far ADIOS 2.x wrote one output file per each writing process. This provides good performance on parallel file systems for up to the point where the file system starts to struggle to handle too many files at once. Aggregation here means that a subset of processes collect the data from all the processes and only they write files. We have implemented the aggregation algorithm known from ADIOS 1.x with some modifications to use less memory than in ADIOS 1.x. At the API level, the user only needs to set the number of “substreams” different from the number of MPI processes. Figure 1 shows an example of how 4 processes can write to a single file. This implementation saturates the MPI network bandwidth while writing to disk in an asynchronous manner. In addition, the new modular architecture of ADIOS2 allows for future experimentation of different aggregation methods to chase performance on upcoming exascale systems.

Multi-Block Writing
ADIOS 2.x now allows writing independent variable pieces to can be later retrieved as a single selection. This is better illustrated in Figure 2, in which each sub-block of data, in blue, is written independently as it becomes available. Whereas a read request can be made for a certain selection, thus pulling data for multiple sub-blocks. This pattern is largely used to support experimental and observational data in which independent I/ O. This feature is naturally integrated into the ADIOS2 API, in which each Variable Put/Get block call is inserted in a container.

This is the third pre-release of the ADaptable Input Output System, ADIOS, version 2 (v2.1.1). This is a beta release with staging support and a stable API intended for users and developers to begin integration and testing of ADIOS 2 into their applications, libraries, and workflows. This version is not intended for production quality purposes.

Major additions from the previous v2.1.0 Beta release include:

* SST Engine for staging data memory-to-memory across applications
* Updated Fortran, C and Python bindings APIs
* Several bugs corrected for default write and read BPFile engine
* Added option for flexible number of substreams (subfiles) based on MPI aggregation for NxM writes (N=mpi processes, M = user defined number of files)
* Added High-Level APIs to each supported language. These APIs are based on the language native IO (C++: adios2::fstream, C: adios2_FILE*, Fortran: file handler, Python: io stream object)
* User Guide documentation moved to readthedocs: [http://adios2-adaptable-io-system-version-2.readthedocs.io/en/latest/](http://adios2-adaptable-io-system-version-2.readthedocs.io/en/latest/)

| **Feature** | ADIOS 1.13 Support | ADIOS 2.1.1 Support |
| ------------- |:-------------:|:-------------:|
| **Write API** | | |
| Local Variable | :white_check_mark: (Offsets, LocalDimensions) | :white_check_mark: (Start, Count) |
| Global Variable | :white_check_mark: (Global) | :white_check_mark: (Variable) |
| Joined Dimensions | :white_check_mark: | :white_check_mark: |
| Dimension Variable | :white_check_mark: | deprecated |
| Constant Dimension Variable | :x: | :white_check_mark: |
| Selection | :white_check_mark: | partial |
| Memory Selection | :white_check_mark: | :x: |
| Attributes | :white_check_mark: | :white_check_mark: |
| Transforms | :white_check_mark: | :x: |
| Config XML | :white_check_mark: | :white_check_mark: |
| Threaded Operations | :x: | :white_check_mark: |
| Time Aggregation | :white_check_mark: | :white_check_mark: |
| Collective Metadata | :white_check_mark: | :white_check_mark: |
| BP3 Format | :white_check_mark: | :white_check_mark: (bitmap stats deprecated) |
| BP4 Format | --- | :x: |
| **I/O Modes** | :white_check_mark: (Transports) | beta |
| BP3 SubFile N-to-N | :white_check_mark: (POSIX) | :white_check_mark: |
| BP3 SubFile N-to-M | :white_check_mark: (MPI_Aggregate) | :white_check_mark: initial support |
| HDF5 | :white_check_mark: | :white_check_mark: |
| WAN (DataMan) | :x: | :white_check_mark: |
| **Read API** | :white_check_mark: | :white_check_mark: |
| **Utils** | | all compatible with ADIOS1.12 |
| bpmeta | :white_check_mark: | :x: |
| bpls | :white_check_mark: | partial bpls2 |
| bpdump | :white_check_mark: | :x: |
| **Language Support** |
| C++ | :white_check_mark: through C | :white_check_mark: native |
| Python | :white_check_mark: Cython | :white_check_mark: PyBind11 |
| C | :white_check_mark: | :white_check_mark: |
| Fortran | :white_check_mark: | :white_check_mark: |
| **Other** |
| Automated testing | :x: | :white_check_mark: (via CTest and GoogleTest) |
| Windows support | :x: | :white_check_mark: (via CTest and GoogleTest) |

This is the second pre-release of the ADaptable Input Output System, ADIOS, version 2 (v2.1.0). This is a beta release with a stable API intended for users and developers to begin integration and testing of ADIOS 2 into their applications, libraries, and workflows. This version is not intended for production quality purposes.

Major additions from the v2.0.0 Alpha release include:

* Re-designed Read and Write APIs for uniformity between the two
* Native read and write support for the ADIOS BP format
* Read and write support for parallel and serial HDF5
* Full support for C, Fortran, and Python APIs in addition to the native C++ interface.
* Initial staging implementations with SST for HPC environments and DataMan for Wide Area Networks.

| **Feature** | ADIOS 1.12 Support | ADIOS 2.1.0 Support |
| ------------- |:-------------:|:-------------:|
| **Write API** | | |
| Local Variable | :white_check_mark: (Offsets, LocalDimensions) | :white_check_mark: (Start, Count) |
| Global Variable | :white_check_mark: (Global) | :white_check_mark: (Variable) |
| Joined Dimensions | :x: | :white_check_mark: |
| Dimension Variable | :white_check_mark: | deprecated |
| Constant Dimension Variable | :x: | :white_check_mark: |
| Selection | :white_check_mark: | partial |
| Memory Selection | :white_check_mark: | :x: |
| Attributes | :white_check_mark: | :white_check_mark: |
| Transforms | :white_check_mark: | :x: |
| Config XML | :white_check_mark: | :white_check_mark: |
| Threaded Operations | :x: | :white_check_mark: |
| Time Aggregation | :white_check_mark: | :white_check_mark: |
| Collective Metadata | :white_check_mark: | :white_check_mark: |
| BP1 Format | :white_check_mark: | :white_check_mark: (bitmap stats deprecated) |
| BP2 Format | --- | :x: |
| **I/O Modes** | :white_check_mark: (Transports) | partial |
| BP1 File N-to-N | :white_check_mark: (POSIX) | :white_check_mark: |
| BP1 File N-to-M | :white_check_mark: (MPI_Aggregate) | :x: |
| HDF5 | :white_check_mark: | :white_check_mark: |
| WAN (DataMan) | :x: | :white_check_mark: |
| **Read API** | :white_check_mark: | :white_check_mark: |
| **Utils** | | all compatible with ADIOS1.12 |
| bpmeta | :white_check_mark: | :x: |
| bpls | :white_check_mark: | partial bpls2 |
| bpdump | :white_check_mark: | :x: |
| **Language Support** |
| C++ | :white_check_mark: through C | :white_check_mark: native |
| Python | :white_check_mark: Cython | :white_check_mark: PyBind11 |
| C | :white_check_mark: | :white_check_mark: |
| Fortran | :white_check_mark: | :white_check_mark: Still to do |
| **Other** |
| Automated testing | :x: | :white_check_mark: (via CTest and GoogleTest) |
| Windows support | :x: | :white_check_mark: (via CTest and GoogleTest) |

Initial release of the ADaptable Input Output System, ADIOS, version 2.0.0. This is an alpha release intended to be distributed so users and developers can test the preliminary list of supported features, provide feedback on the source code, documentation, API, and development flows, etc., and report bugs. This version is not intended for production quality purposes.

Please see [Getting Started](https://github.com/ornladios/ADIOS2/blob/master/ReadMe.md) for instructions on how to configure and build this release, along with the [User Guide](https://github.com/ornladios/ADIOS2/wiki/User-Guide-v2.0.0-June-30th-2017).

|**Summary** | |
|:------------- |----------------------------|
| Supported 64-bit OS | Linux |
| | Mac OS X El Capitan |
| | Windows 10 |