Adios2

Summary
ADIOS v2.5.0, September 30th 2019 This release should be considered stable and suitable for deployment. We highly recommend migration to this version for long term support and future API compatibility. This is a major update which includes the introduction of a new directory-based file format, BP4, for the default Engine targeting streaming through files applications.

Features
* Virtual Engines (File, FileStream, InSituAnalysis, InSituVisualization, CodeCoupling)
* BP4 file engine has a constant append time of new steps. Faster than creating new output at every step.
* Default is now the BP4 file engine that supports reading streaming concurrently while the writer is still producing new steps
* Zero-copy access to the engine buffer memory in Write mode. Uses Variable<T>::Span per Engine Put block, based on upcoming [C++20 std::span](https://en.cppreference.com/w/cpp/container/span). Available in BP3 (since v2.4.0) and BP4 (since v2.5.0) Engines C++ APIs only.

API Breakage
* Python high-level API: ability to retrieve single global values as numpy's 0-dimension array **This breaks API compatibility with v2.4.0 Python high-level API**

Summary
ADIOS v2.4.0, June 30th 2019. This release should be considered stable and suitable for deployment. We highly recommend migration to this version for long term support and future API compatibility.

Features
* Automatic conversion to fixed width types (`int` -> `int32_t`), in all bindings to facilitate portable types and `bpls` output
* MPI library "serial" functionality. ADIOS2 library compiled with MPI can run with code not invoking `MPI_Init` and `MPI_Finalize` running in serial. Requires linking with MPI for symbols.
* Stable high-level APIs in Python and C++ supporting local variables
* Zero-copy access to the engine buffer memory in Write mode. Uses `Variable<T>::Span` per Engine Put block, based on upcoming [C++20 std::span](https://en.cppreference.com/w/cpp/container/span). Available in BP3 Engine C++ APIs only.
* New functionality in Fortran and C APIs for safer handling of subroutines/functions outputting strings
* Addition of no-op backends to disable selected functionality and / or evaluate the performance impact of various layers
- `null` Engine : Bypass all library actions as soon as possible, often in the high-level language bindings before the core library is even reached
- `nullcore` Engine: Implemented in the core of the library allowing the front-end data management and variable operations to occur but to transparently ignore all write and buffering requests.
- `null` Transport : For use with the BP engines, this will bypass all disk I/O operations. All of the data management and buffering will still occur but there will be no actual data written to a file.
* Optimization of data exchanges in the SST engine when a fixed data exchange pattern is specified via LockWriterDefinitions/LockReaderSelections
* PreciousFirstTimestep engine mode allows saving mesh and other one-time-output in a stream, so that late-comer readers can properly initialize from the first input step. Available in SST engine only.
* Added support for MemorySelection for reading into non-contiguous memory (e.g. ghost cells). Available in BP3 Engine only.
* Fixed several bugs: substreams aggregation, endian interoperability, local variables and `bpls`
* Complete redesign of [the user guide documentation in readthedocs](https://adios2.readthedocs.io)
* Added several new lossless compression plugins in BP buffer: `bzip2`, `libpng`, `blosc`. Available in BP and SST engines only.
* Availability of tools interface with connection to TAU 2.28 or greater.

Changes

* LatestAvailable reading mode has been removed from BeginStep() and is now specified at Open() with AlwaysProvideLatestTimestep engine parameter. This allows for more optimized queueing behavior in staging engines. **This breaks API compatibility with v2.3.x**

* IO.LockDefinitions() function has been removed and replaced with Engine.LockWriterDefinitions() and Engine.LockReaderSelections() to improve clarity and separate functionality. As before, calling these functions informs ADIOS2 that the write/read pattern between two applications in staging is unchanging from timestep to timestep (after the calls). This allows certain optimizations, such as eliminating some metadata exchanges and pre-loading data to the readers. **This breaks API compatibility with v2.3.x**

* Data queueing in the SST engine has been redesigned, adding more explicit support for retaining timesteps for late-arriving readers (see the ReserveQueueLimit engine parameter), but also changing the way timesteps are handled when there are no readers present. In the no-readers case, specifying QueueFullPolicy:Blocking will no longer cause SST to pause until a reader arrives. **This behavior is a change from v2.3.x**

Summary
This is a release candidate for the upcoming v2.4.0 release in June 30th 2019. The v2.4.0-rc1 release should be considered stable and suitable for testing deployment as a module on HPC platforms. We recommend users to migrate to this version to test for the upcoming v2.4.0 and report any outstanding issue.

Features
* Moved completely to fix width types in all bindings to facilitate portable types and bpls output
* Stable high-level APIs for Python and C++ supporting local variables
* Added NULL Engine to measure Engine overall overhead
* `Variable<T>::Span` provide access to the engine buffer memory in Write mode (C++ APIs) similar to C++20 `std::span` of a `std::vector`
* New functionality in Fortran and C APIs for handling strings in a safer mode
* Fixed several bugs: Substreams aggregation, endian interoperability, bpls
* Update documentation: adios2.readthedocs.io

Summary
This is a bug-fix release on top of the `v2.3.0` release. The v2.3.1 release should be considered stable and suitable for deployment as a module on HPC platforms.

Features
Bug fixes beyond the `v2.3.0` release include:
* Various CMake fixes
* 1203 1204
* Statically generate adios2-config at install time so cmake and python are no longer required
* 1172 1191
* Various bpls bug fixes
* 1175 1182 1211
* Add missing attributes to high level API
* 1146 1197
* Various SST bug fixes
* 1157 1167 1152 1151 1149 1186 1183 1201 1207 1213
* HDF5 Single value attributes
* 1159
* MGARD tweaks
* 1196

Summary
This is the first production release of the ADIOS v2.x branch. At this point, all public APIs can be considered fairly stable. This also marks the official transition from ADIOS v1.x to v2.x, thus v1.x should now be considered deprecated.

Features
Major additions from the previous v2.2 Beta release include:

* Additional testing added reaching over 300 test covering both low level unit tests and higher level mini-app use-cases.
* APIs for local array handling (each rank is an independent chunk of data instead of a smaller piece in a global index space)
* The dissemination of per-rank write block information so a reader can know which pieces of a variable were written by which rank and optimize their read selection accordingly.
* An extensive list of bug fixes.

Summary
This is the fourth pre-release of the ADaptable Input Output System, ADIOS, version 2 (v2.2.0). The primary focus of this beta release is increased performance through I/O aggregation. This version is not intended for production quality purposes.

Features
Major additions from the previous v2.1.1 Beta release include process aggregation and multi-block I/O.

Aggregation
So far ADIOS 2.x wrote one output file per each writing process. This provides good performance on parallel file systems for up to the point where the file system starts to struggle to handle too many files at once. Aggregation here means that a subset of processes collect the data from all the processes and only they write files. We have implemented the aggregation algorithm known from ADIOS 1.x with some modifications to use less memory than in ADIOS 1.x. At the API level, the user only needs to set the number of “substreams” different from the number of MPI processes. Figure 1 shows an example of how 4 processes can write to a single file. This implementation saturates the MPI network bandwidth while writing to disk in an asynchronous manner. In addition, the new modular architecture of ADIOS2 allows for future experimentation of different aggregation methods to chase performance on upcoming exascale systems.

Multi-Block Writing
ADIOS 2.x now allows writing independent variable pieces to can be later retrieved as a single selection. This is better illustrated in Figure 2, in which each sub-block of data, in blue, is written independently as it becomes available. Whereas a read request can be made for a certain selection, thus pulling data for multiple sub-blocks. This pattern is largely used to support experimental and observational data in which independent I/ O. This feature is naturally integrated into the ADIOS2 API, in which each Variable Put/Get block call is inserted in a container.