Teca

Major features
===============
The TECA data model now supports memory management on CPUs as well as CUDA,
OpenMP device offload, HIP capable GPUs and accelerators.
TECA's execution model was extended to support CUDA capable GPUs. This includes
automated load balancing across multi GPU accelerated compute nodes on
supercomputing systems as well as CUDA kernel launching and load balancing
infrastructure
Support for zero-copy interpoerability with Cupy and Numba on CUDA capable
GPU's was added.

GPUized algorithms
==================
teca_binary_segmentation
teca_l2_norm
teca_valid_value_mask
teca_unpack_data
teca_integrated_vapor_transport
teca_temporal_reduction
teca_lapse_rate
teca_cf_reader
teca_cf_writer

New algorithms and apps
=======================
teca_lapse_rate
teca_tc_potential_intensity
teca_time_axis_convolution
teca_shapefile_mask
teca_tempest_remap
teca_cartesian_mesh_coordinate_transform
teca_array_collection_reader
teca_array_collection_writer

Improvements
==============
Make the teca_array_collection a data set
Add user defined intervals and operators to the teca_temporal_reduction
teca_temporal_reduction handle integer data in the avergaing reduction
teca_temporal_reduction use the valid value mask
add a summation reduction to the teca_temporal_reduction
improved threading support on MacOS
users can provide call backs at runtime for custom error handling

Documentation
=============
Numerous improvements to the user guide and Doxygen documentation including
documentation of new applications and install on GPU enabled systems
Updated examples illustrating how to use Cupy in Python applications
New Perlmutter specific examples were added to TECA_Examples

4.1.0 is a feature release with a number of new and exciting features and a number of critical bug fixes.

* new mask below surface algorithm that creates point wise binary (0,1) mask identifying mesh points that are below land surface based on externally provided DEM.
* integrated the mask below surface stage into the BARD, IWV, and IVT apps
* new unpack NetCDF packed data stage
* add coordinate normalization stage transform for longitude from -180 to 180
to 0 to 360
* new IWV algorithm
* new IWV command line application
* new time based file layouts (daily, monthly, yearly, seasonal)
* BARD app can now generate output fields weighted by AR probabilities
* new rename variables stage
* improvements to cartesian_mesh_source for remeshing
* cf_reader correctly detects centering and per field dimensionality
* multi_cf_reader MCF file format improvements. Add support for reader
properties, globablly and per reader.
* cf_reader option to produce 2D field when the 3'rd dimension is length 1
* Cartesian meshes can now contain both 2D and 3D arrays, metadata annotations
are used to differentiate at run time
* metadata probe improvements to report per-field centering
* new remeshing capability deployed in cf_restripe and apps that utilize
elevation mask
* improvements to the user guide
* refactored source code documentation to be compatible with Doxygen,
* published Doxygen on the rtd site : https://teca.readthedocs.io/en/integrating_breathe/doxygen/index.html
* new capabilities in the cf_restripe command line application for remeshing
* 25+ bug fixes

Documentation

1. A major overhaul of the command line application section of the user guide including the addition of examples.
1. Publish batch scripts illustrating running TECA at scale in the new [TECA_examples](https://github.com/LBL-EESA/TECA_examples) repo.
1. Giving tutorials and publishing the materials in the new [TECA_tutorials](https://sourceforge.net/p/teca/TECA_tutorials) repo
1. Updates to the installation section of the TECA User's Guide](https://teca.readthedocs.io/en/latest/installation.html)


Data Model Improvements

1. Added support for Arakawa C Grids in `teca_arakawa_c_grid`
1. Added support for logically Cartesian so called curvilinear grids in `teca_curvilinear_mesh`
1. Refactored mesh related class hierarchy so that common codes such as array accessing and I/O live in `teca_mesh`
1. Added support for face and edge centered mesh based data.

I/O Capabilities

1. Added reader for WRF simulation `teca_wrf_reader`
1. Add support for writing logically Cartesian curvilinear meshes in `teca_cartesian_mesh_writer`.
1. Added a new NetCDF based output format for tabular data to the `teca_table_writer`.
1. Added support for reading tabular CSV files to the `teca_table_reader`. This enables the tabular outputs such as TC tracks etc saved from TECA apps to be stored in a format ingestible by other tools such as Python and Excel without the need to convert from TECA's internal binary format.
1. Added versioning and error checking to TECA's internal binary serialization format across all datasets. This enables us to catch version differences and handle bad or corrupted files gracefully.
1. use of NetCDF parallel 4 (i.e. MPI collective I/O) for writing results. this enables the use of any number of files with any number of ranks.

Execution Patterns

1. Implement a new streaming mode reduction where data is incrementally reduced as it becomes available. This parallelizes the reduction step and reduces the memory overhead.
1. Introducing a new MPI parallel approach to scan the time axis. This has substantial benefit when there are a large number of files.
1. expose MPI aware thread load balancing to Python. This was used in the `teca_pytorch_algorithm` to automatically load balance the OpenMP backend of PyTorch.
1. implement GPU load balancing strategy in the `teca_pytorch_algorithm`.
1. Enable process groups to be excluded from execution. This lets a pipeline run on a subset of MPI_COMM_WORLD.

Algorithmic Capabilities

1. Added `teca_pytorch_algorithm` a base class that handle tasks common to interfacing to PyTroch when developing Machine Learning based detectors.
1. Added `teca_deeplab_ar_detect` a new PyTorch based Machine Learning based AR detector.
1. Added `teca_valid_value_mask` an algorithm that generates a mask identifying the presence of NetCDF \_FillValue values in arrays. Down stream algorithms use the mask to handle \_FillValue's in an algorithm appropriate manner.
1. Added `teca_temporal_reduction` an algorithm that implements transformations from one time resolution to another. The implementation includes min, max, and average operators and supports daily, monthly, and seasonal intervals.
1. Added `teca_vertical_reduction` an algorithm that converts 3D data to 2D by applying a reduction in the vertical spatial dimension. This is a base class that contains code common to vertical reductions.
1. Added `teca_integrated_vapor_transport` a vertical reduction that computes IVT from horizontal wind vector and specific humidity.
1. An improved floating point differencing algorithm was developed and a number of codes were updated to use it.

Command Line Applications

1. Added `teca_integrated_vapor_transport` command line application for computing IVT.
1. Added `teca_restripe` command line application for re-organizing NetCDF datasets.
1. Added `teca_deeplab_ar_detector` command line application detecting AR's using machine learning.
1. Integrated IVT calculations into the `teca_nayesian_ar_detector`.
1. Normalized names and meaning of command line options across command line applications

Python Capabilities

1. A polymorphic redesigned the `teca_python_algorithm` makes it easier to use.
1. Handle numpy scalar types
1. Expose more features such as MPI aware thread load balancing, calendaring, profiling, and file manipulation utilities.

Testing

1. Added testing infrastructure and tests for command line applications
1. Deployed testing on Ubuntu 18.04, Fedora 31, Fedora 32, and Mac OS with xcode 12.2.

Bug fixes

More than 50 bug fixes were reported.

This is a major release in support of:

T.A. O'Brien et al, **"Detection of Atmospheric Rivers with Inline
Uncertainty Quantification: TECA-BARD v1.0"**, Geoscientific Model
Development, submitted winter 2020

The pipeline internals were refactored to be more general, the assumption that
time was the dimension across which the reduction is applied was removed, as
well as changes that enable nested map-reduce.

The TECA User Guide was ported to "Read the Docs". https://teca.readthedocs.io

Our Travis CI test infrastructure was updated to use Docker, and two new OS
images Fedora 28, and Ubuntu 18.04 were deployed.

More than 40 bug fixes

New algorithms included in this release:

| Type | Name | Description |
|---------------------|------------------------------------|-----------------------------------------------------------------------------------|
| general puprose | teca_2d_component_area | Computes the area's of regions identified by the connected components filter. |
| general puprose | teca_bayesian_ar_detect | Detects atmospheric rivers using a Bayesian method. |
| general puprose | teca_bayesian_ar_detect_parameters | Parameters used by Bayesian AR detector. |
| general puprose | teca_cartesian_mesh_source | Used to create Cratesian meshes in memory and inject them into a pipeline. |
| general puprose | teca_component_area_filter | Masks regions with area out side a user specified range |
| general puprose | teca_component_statistics | Gathers information about connected component regions into a tabular format |
| general puprose | teca_latitude_damper | Multiplies a field by an inverted Gaussian (user specified mean and HWHM) |
| general puprose | teca_normalize_coordinates | Transforms Cartesian meshes such that coordinates are always in ascending order |
| general puprose | teca_python_algorithm | Base class for TECA algorithm's written in Python. Handles internal plumbing |
| core infrastructure | teca_memory_profiler | Supporting class that samples memory consumtion during application execution |
| core infrastructure | teca_profiler | Supporting class that logs start, stop, and duration of developer defined events |
| I/O | teca_cartesian_mesh_reader | Reads TECA Cartesian meshes in TECA's internal binary format |
| I/O | teca_cartesian_mesh_writer | Writes TECA Cartesian meshes in TECA's internal binary format |
| I/O | teca_cf_writer | Writes TECA Cratesian meshes in NetCDF CF2 conventions |

New applications included in this release:

| Name | Description |
|--------------------------|-------------------------------------------------------------------------|
| teca_bayesian_ar_detect | Command line application that can be used to detect AR's on HPC systems |
| teca_profile_explorer | Interactive tool for exploring run time profiling data |