Devito

GPUs
* Support for more Operators -- isotropic acoustic, TTI, elastic, viscoelastic, ...
* Support for reductions
* Fixed several bugs concerning code generation

DSL
* Fixed minor bugs related to derivatives and finite-differences

MPI
* Fixed several bugs arising in presence of SubDomains and/or SubDimensions
* Fixed code generation in full mode (compute-communication overlap), when the same symbol (e.g. `comm`) was appearing twice in the signature of elemental functions

Documentation
* Updated user API
* Additional tutorials in the form of Jupyter notebooks (SymPy, DSL, SubDomains)

Benchmarking
* A new mode exploiting the JIT_BACKDOOR mechanism was added
* DevitoBench package merged into Devito

CI/CD:
* Dropped Azure Pipelines and Travis; testing infrastructure moved to GitHub Actions
* Added GPU testing
* Added testing for osx
* Auto-deployment on Pypi upon release

Compiler:
* Improved architecture auto-detection
* Refactored DLE as a sequence of "standard" compilation passes
* Moved data dependence analysis to the Cluster level
* Fixed data dependence analysis in some corner cases arising with SubDomains and/or SubDimensions
* Moved interpolation to separate sub-package
* Integrated profiling for the compilation pipeline

Many thanks to all the contributors to this release (last surname alphabetical order):
* Tarek Allam (The Alan Turing Institute)
* George Bisbas (Imperial College London)
* Maelso Bruno (Federal University of Paraíba)
* Tim Burgess (DownUnder Geosolutions)
* Gerard Gorman (Imperial College London)
* Navjot Kukreja (Imperial College London)
* Mathias Louboutin (Georgia Institute of Technology)
* Fabio Luporini (Devito Codes)
* Rhodri Nelson (Imperial College London)
* BlockSprintZIf (Independent)

Tensor algebra support (873):
- `VectorFunction` and `VectorTimeFunction`
- (2nd order) `TensorFunction` and `TensorTimeFunction`
- Full support for FD and related operations (derivatives, shortcuts, solve, ...)
- Differential operators such as `div`, `grad` and `curl`

FD extensions:
- Custom FD with user-supplied coefficients as `Function` (964)

Extended and more rigorous support for staggered grids (873):
- True half-grid staggering (`u(x + h_x/2)`)
- Automatic evaluation at half-nodes (averaging only)
- Automatic staggered FD of any order

MPI:
- MPI data slicing fixes(1008)
- SparseFunction fixes (gathering with correct coordinates) (922)
- MPI parameters correctly passed to the `Operator` (1016, 1019)
- Gathering bug fix (916)
- Domain decomposition improvements (915, 904)
- Effective computation/communication overlap
- Fixed argument processing to acquire the correct MPI communicator

GPU:
- More progress on the OPS backend
- OpenMP-5.x offloading experimental support (1004, 1012)
(note: compiler installation instructions available on the Devito wiki)

Performance:
- Better OpenMP scheduling and sparse loop handling (995)
- Faster compilation (932)

Misc:
- More robust inheritance from `sympy` and support for latest version (1005)
- `Model` bug fixes and cleanup (1000, 1015)
- `Model` now initializes/pads all object via Operators (911)
- `CustomCompiler` now uses the system/user provided parameters correctly (925)
- Devito based Gaussian smoother (931)
- ARM support (940)
- Better caching infrastructure (956) removing the majority of memory leaks
- Improved benchmarking infrastructure

Many thanks to all the contributors to this release (last surname alphabetical order):

* George Bisbas (Imperial College London)
* Maelso Bruno (Federal University of Paraíba)
* Lucas Cavalcante (Federal University of Rio Grande do Norte)
* Gabriel Sebastian von Conta (Universidade Federal do Rio Grande do Norte)
* Arthur Giannotta (University of São Paulo)
* Gerard Gorman (Imperial College London)
* Navjot Kukreja (Imperial College London)
* Mathias Louboutin (Georgia Institute of Technology)
* Fabio Luporini (Imperial College London)
* Vitor Mickus (Federal University of Rio Grande do Norte)
* Rhodri Nelson (Imperial College London)
* Vincenzo Pandolfo (G-Research)
* Elias Gabriel Amaral da Silva (Federal University of Rio Grande do Norte)
* Jaime Freire de Souza (University of São Paulo)

Release notes
--------------

* MPI support:
- Python-level: MPI-distributed NumPy arrays.
- C-level: code generation for sub-domains, staggered grids, operators with coupled PDEs.
- C-level: performance optimizations (e.g., computation-communication overlap).
* Lazy evaluation of derivatives.
* Revisited staggered grids API (now Dimension-based, previously mask-based).
* Re-engineered clustering (which means smarter loop fusion/fission).
* DSE: Improved aliases detection.
* DLE: OpenMP nested parallelism; hierarchical loop blocking.
* Auto-padding for Functions/TimeFunctions.
* Improved data dependency analysis.
* Smarter Operator auto-tuning.
* New tutorials: Operator application, MPI, new propagators, custom stencils, and more.
* Revisited benchmarking scripts.
* Revisited examples, new models and propagators (e.g., visco-elastic).
* Smarter continuous integration: now Travis sided by Azure Pipelines; dropped Jenkins.
* Misc bug fixes.
* Hundreds of tests added.
* More sophisticated platform auto-detection.
* Update in checkpointing (via PyRevolve) to significantly reduce memory consumption and overall runtimes. Not backward compatible.
* Experimental support for lossy compression of checkpoints through PyRevolve + ZFP.

Release notes
--------------

* Preliminary support for MPI (no changes to user code requested)
* Support for staggered grids
* Improved compilation technology
* Improved Operator autotuning
* More powerful DSL (e.g., take derivatives of entire expressions such as `(u+v).dx`)
* More efficient pickling
* Misc bug fixes
* New modeling examples based on the elastic wave equation
* New examples describing aspects of the compilation technology

This release ships the following features, besides a large number of misc improvements and bug fixes

* Support for Docker-based installation
* Introduction of PrecomputedSparseFunction for SparseFunction with precomputed coefficients
* Smarter JIT engine, including autodetection of the backend-compiler version for ad-hoc optimisations
* Operator pickling
* Introduction of new IR level: the ScheduleTree level
* Smarter loop generation
* Better, more efficient data dependence analysis
* Caching of JIT-compiled code
* More accurate profiling reports
* Extended YASK support (e.g., boundary conditions)

dft-paper

This release builds upon ``v3.1.0`` including all features from Devito ``v3.2-beta`` as well as the following changes:

* Domain-allocation switch: now Operators run over the entire Grid, no matter what the stencil radius is.
* "Min-max" API to iterate over subregions of the Grid.
* A set of examples showing how to use Devito for linear algebra calculations
* Better seismic examples, more resampling.
* Parallelization of loops with reduction dependences with OpenMP atomics
* Simpler, better, more robust processing for Operator arguments
* Significantly more powerful IterationSpace
* Smarter clusterization