Esmvalcore

------
Highlights
~~~~~~~~~~

- We have a new preprocessor function called `'rolling_window_statistics' <https://docs.esmvaltool.org/projects/ESMValCore/en/latest/recipe/preprocessor.html#rolling-window-statistics>`__ implemented by :user:`malininae`
- We have improved the support for native models, refactored native model fixes by adding common base class `NativeDatasetFix`, changed default DRS for reading native ICON output, and added tests for input/output filenames for `ICON <https://docs.esmvaltool.org/projects/ESMValCore/en/latest/quickstart/find_data.html#icon>`__ and `EMAC <https://docs.esmvaltool.org/projects/ESMValCore/en/latest/quickstart/find_data.html#emac>`__ on-the-fly CMORizer, all these features courtesy of :user:`schlunma`
- Performance of preprocessor functions that use time dimensions has been sped up by **two orders of magnitude** thanks to contributions by :user:`bouweandela`

This release includes:

Backwards incompatible changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Change default DRS for reading native ICON output (:pull:`1705`) by :user:`schlunma`

Bug fixes
~~~~~~~~~

- Add support for regions stored as MultiPolygon to extract_shape preprocessor (:pull:`1670`) by :user:`bouweandela`
- Fixed type annotations for Python 3.8 (:pull:`1700`) by :user:`schlunma`
- Core `_io.concatenate()` may fail due to case when one of the cubes is scalar - this fixes that (:pull:`1715`) by :user:`valeriupredoi`
- Pick up esmvalcore badge instead of esmvaltool one in README (:pull:`1749`) by :user:`valeriupredoi`
- Restore support for scalar cubes to time selection preprocessor functions (:pull:`1750`) by :user:`bouweandela`
- Fix calculation of precipitation flux in EMAC on-the-fly CMORizer (:pull:`1755`) by :user:`schlunma`

Deprecations
~~~~~~~~~~~~

- Remove deprecation warning for regrid schemes already deprecated for v2.7.0 (:pull:`1753`) by :user:`valeriupredoi`

Documentation
~~~~~~~~~~~~~

- Add Met Office Installation Method (:pull:`1692`) by :user:`mo-tgeddes`
- Add MO-paths to config file (:pull:`1709`) by :user:`mo-tgeddes`
- Update MO obs4MIPs paths in the user configuration file (:pull:`1734`) by :user:`mo-tgeddes`
- Update `Making a release` section of the documentation (:pull:`1689`) by :user:`sloosvel`
- Added changelog for v2.7.0 (:pull:`1746`) by :user:`valeriupredoi`
- update CITATION.cff file with 2.7.0 release info (:pull:`1757`) by :user:`valeriupredoi`

Improvements
~~~~~~~~~~~~

- New preprocessor function 'rolling_window_statistics' (:pull:`1702`) by :user:`malininae`
- Remove `pytest_flake8` plugin and use `flake8` instead (:pull:`1722`) by :user:`valeriupredoi`
- Added CESM2 CMORizer (:pull:`1678`) by :user:`schlunma`
- Speed up functions that use time dimension (:pull:`1713`) by :user:`bouweandela`
- Modernize and minimize pylint configuration (:pull:`1726`) by :user:`bouweandela`

Fixes for datasets
~~~~~~~~~~~~~~~~~~

- Refactored native model fixes by adding common base class `NativeDatasetFix` (:pull:`1694`) by :user:`schlunma`

Installation
~~~~~~~~~~~~

- Pin `netCDF4 != 1.6.1` since that seems to throw a flurry of Segmentation Faults (:pull:`1724`) by :user:`valeriupredoi`

Automatic testing
~~~~~~~~~~~~~~~~~

- Pin `flake8<5.0.0` since Circle CI tests are failing copiously (:pull:`1698`) by :user:`valeriupredoi`
- Added tests for input/output filenames for ICON and EMAC on-the-fly CMORizer (:pull:`1718`) by :user:`schlunma`
- Fix failed tests for Python<3.10 resulting from typing (:pull:`1748`) by :user:`schlunma`

.. _changelog-v2-6-0:

------

Highlights
~~~~~~~~~~

- A new set of CMOR fixes is now available in order to load native EMAC model output and CMORize it on the fly. For details, see :ref:`Supported native models: EMAC <read_emac>`.
- The version number of ESMValCore is now automatically generated using `setuptools_scm <https://github.com/pypa/setuptools_scm/#default-versioning-scheme>`__, which extracts Python package versions from git metadata.

This release includes

Deprecations
~~~~~~~~~~~~

- Deprecate the function `esmvalcore.var_name_constraint` (:pull:`1592`) by :user:`schlunma`. This function is scheduled for removal in v2.8.0. Please use :class:`iris.NameConstraint` with the keyword argument `var_name` instead: this is an exact replacement.

Bug fixes
~~~~~~~~~

- Added `start_year` and `end_year` attributes to derived variables (:pull:`1547`) by :user:`schlunma`
- Show all results on recipe results webpage (:pull:`1560`) by :user:`bouweandela`
- Regridding regular grids with similar coordinates (:pull:`1567`) by :user:`tomaslovato`
- Fix timerange wildcard search when deriving variables or downloading files (:pull:`1562`) by :user:`sloosvel`
- Fix `force_derivation` bug (:pull:`1627`) by :user:`sloosvel`
- Correct `build-and-deploy-on-pypi` action (:pull:`1634`) by :user:`sloosvel`
- Apply `clip_timerange` to time dependent fx variables (:pull:`1603`) by :user:`sloosvel`
- Correctly handle requests.exceptions.ConnectTimeout when an ESGF index node is offline (:pull:`1638`) by :user:`bouweandela`

CMOR standard
~~~~~~~~~~~~~

- Added custom CMOR tables used for EMAC CMORizer (:pull:`1599`) by :user:`schlunma`
- Extended ICON CMORizer (:pull:`1549`) by :user:`schlunma`
- Add CMOR check exception for a basin coord named sector (:pull:`1612`) by :user:`dhohn`
- Custom user-defined location for custom CMOR tables (:pull:`1625`) by :user:`schlunma`

Containerization
~~~~~~~~~~~~~~~~

- Remove update command in Dockerfile (:pull:`1630`) by :user:`sloosvel`

Community
~~~~~~~~~

- Add David Hohn to contributors' list (:pull:`1586`) by :user:`valeriupredoi`

Documentation
~~~~~~~~~~~~~

- [Github Actions Docs] Full explanation on how to use the GA test triggered by PR comment and added docs link for GA hosted runners (:pull:`1553`) by :user:`valeriupredoi`
- Update the command for building the documentation (:pull:`1556`) by :user:`bouweandela`
- Update documentation on running the tool (:pull:`1400`) by :user:`bouweandela`
- Add support for DKRZ-Levante (:pull:`1558`) by :user:`remi-kazeroni`
- Improved documentation on native dataset support (:pull:`1559`) by :user:`schlunma`
- Tweak `extract_point` preprocessor: explain what it returns if one point coord outside cube and add explicit test (:pull:`1584`) by :user:`valeriupredoi`
- Update CircleCI, readthedocs, and Docker configuration (:pull:`1588`) by :user:`bouweandela`
- Remove support for Mistral in `config-user.yml` (:pull:`1620`) by :user:`remi-kazeroni`
- Add changelog for v2.6.0rc1 (:pull:`1633`) by :user:`sloosvel`
- Add a note on transferring permissions to the release manager (:pull:`1645`) by :user:`bouweandela`
- Add documentation on building and uploading Docker images (:pull:`1644`) by :user:`bouweandela`
- Update documentation on ESMValTool module at DKRZ (:pull:`1647`) by :user:`remi-kazeroni`
- Expanded information on deprecations in changelog (:pull:`1658`) by :user:`schlunma`

Improvements
~~~~~~~~~~~~

- Removed trailing whitespace in custom CMOR tables (:pull:`1564`) by :user:`schlunma`
- Try searching multiple ESGF index nodes (:pull:`1561`) by :user:`bouweandela`
- Add CMIP6 `amoc` derivation case and add a test (:pull:`1577`) by :user:`valeriupredoi`
- Added EMAC CMORizer (:pull:`1554`) by :user:`schlunma`
- Improve performance of `volume_statistics` (:pull:`1545`) by :user:`sloosvel`

Fixes for datasets
~~~~~~~~~~~~~~~~~~

- Fixes of ocean variables in multiple CMIP6 datasets (:pull:`1566`) by :user:`tomaslovato`
- Ensure lat/lon bounds in FGOALS-l3 atmos variables are contiguous (:pull:`1571`) by :user:`sloosvel`
- Added `AllVars` fix for CMIP6's ICON-ESM-LR (:pull:`1582`) by :user:`schlunma`

Installation
~~~~~~~~~~~~

- Removed `package/meta.yml` (:pull:`1540`) by :user:`schlunma`
- Pinned iris>=3.2.1 (:pull:`1552`) by :user:`schlunma`
- Use setuptools-scm to automatically generate the version number (:pull:`1578`) by :user:`bouweandela`
- Pin cf-units to lower than 3.1.0 to temporarily avoid changes within new version related to calendars (:pull:`1659`) by :user:`valeriupredoi`

Preprocessor
~~~~~~~~~~~~

- Allowed special case for unit conversion of precipitation (`kg m-2 s-1` <--> `mm day-1`) (:pull:`1574`) by :user:`schlunma`
- Add general `extract_coordinate_points` preprocessor (:pull:`1581`) by :user:`sloosvel`
- Add preprocessor `accumulate_coordinate` (:pull:`1281`) by :user:`jvegreg`
- Add `axis_statistics` and improve `depth_integration` (:pull:`1589`) by :user:`sloosvel`

Release
~~~~~~~

- Increase version number for ESMValCore v2.6.0rc1 (:pull:`1632`) by :user:`sloosvel`
- Update changelog and version for 2.6rc3 (:pull:`1646`) by :user:`sloosvel`
- Add changelog for rc4 (:pull:`1662`) by :user:`sloosvel`


Automatic testing
~~~~~~~~~~~~~~~~~

- Refresh CircleCI cache weekly (:pull:`1597`) by :user:`bouweandela`
- Use correct cache restore key on CircleCI (:pull:`1598`) by :user:`bouweandela`
- Install git and ssh before checking out code on CircleCI (:pull:`1601`) by :user:`bouweandela`
- Fetch all history in Github Action tests (:pull:`1622`) by :user:`sloosvel`
- Test Github Actions dashboard badge from meercode.io (:pull:`1640`) by :user:`valeriupredoi`
- Improve esmvalcore.esgf unit test (:pull:`1650`) by :user:`bouweandela`

Variable Derivation
~~~~~~~~~~~~~~~~~~~

- Added derivation of `hfns` (:pull:`1594`) by :user:`schlunma`

.. _changelog-v2-5-0:

------

Highlights
~~~~~~~~~~

- The new preprocessor :func:`~esmvalcore.preprocessor.extract_location` can extract arbitrary locations on the Earth using the `geopy <https://pypi.org/project/geopy/>`__ package that connects to OpenStreetMap. For details, see :ref:`Extract location <extract_location>`.
- Time ranges can now be extracted using the `ISO 8601 format <https://en.wikipedia.org/wiki/ISO_8601>`_. In addition, wildcards are allowed, which makes the time selection much more flexible. For details, see :ref:`Recipe section: Datasets <Datasets>`.
- The new preprocessor :func:`~esmvalcore.preprocessor.ensemble_statistics` can calculate arbitrary statistics over all ensemble members of a simulation. In addition, the preprocessor :func:`~esmvalcore.preprocessor.multi_model_statistics` now accepts the keyword ``groupy``, which allows the calculation of multi-model statistics over arbitrary multi-model ensembles. For details, see :ref:`Ensemble statistics <ensemble statistics>` and :ref:`Multi-model statistics <multi-model statistics>`.

This release includes

Backwards incompatible changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Update Cordex section in `config-developer.yml` (:pull:`1303`) by :user:`francesco-cmcc`. This changes the naming convention of ESMValCore's output files from CORDEX dataset. This only affects recipes that use CORDEX data. Most likely, no changes in diagnostics are necessary; however, if code relies on the specific naming convention of files, it might need to be adapted.
- Dropped Python 3.7 (:pull:`1530`) by :user:`schlunma`. ESMValCore v2.5.0 dropped support for Python 3.7. From now on Python >=3.8 is required to install ESMValCore. The main reason for this is that conda-forge dropped support for Python 3.7 for OSX and arm64 (more details are given `here <https://github.com/ESMValGroup/ESMValTool/issues/2584#issuecomment-1063853630>`__).

Bug fixes
~~~~~~~~~

- Fix `extract_shape` when fx vars are present (:pull:`1403`) by :user:`sloosvel`
- Added support of `extra_facets` to fx variables added by the preprocessor (:pull:`1399`) by :user:`schlunma`
- Augmented input for derived variables with extra_facets (:pull:`1412`) by :user:`schlunma`
- Correctly use masked arrays after `unstructured_nearest` regridding (:pull:`1414`) by :user:`schlunma`
- Fixing the broken derivation script for XCH4 (and XCO2) (:pull:`1428`) by :user:`hb326`
- Ignore `.pymon-journal` file in test discovery (:pull:`1436`) by :user:`valeriupredoi`
- Fixed bug that caused automatic download to fail in rare cases (:pull:`1442`) by :user:`schlunma`
- Add new `JULIA_LOAD_PATH` to diagnostic task test (:pull:`1444`) by :user:`valeriupredoi`
- Fix provenance file permissions (:pull:`1468`) by :user:`bouweandela`
- Fixed usage of `statistics=std_dev` option in multi-model statistics preprocessors (:pull:`1478`) by :user:`schlunma`
- Removed scalar coordinates `p0` and `ptop` prior to merge in `multi_model_statistics` (:pull:`1471`) by :user:`axel-lauer`
- Added `dataset` and `alias` attributes to `multi_model_statistics` output (:pull:`1483`) by :user:`schlunma`
- Fixed issues with multi-model-statistics timeranges (:pull:`1486`) by :user:`schlunma`
- Fixed output messages for CMOR logging (:pull:`1494`) by :user:`schlunma`
- Fixed `clip_timerange` if only a single time point is extracted (:pull:`1497`) by :user:`schlunma`
- Fixed chunking in `multi_model_statistics` (:pull:`1500`) by :user:`schlunma`
- Fixed renaming of auxiliary coordinates in `multi_model_statistics` if coordinates are equal (:pull:`1502`) by :user:`schlunma`
- Fixed timerange selection for automatic downloads (:pull:`1517`) by :user:`schlunma`
- Fixed chunking in `multi_model_statistics` (:pull:`1524`) by :user:`schlunma`

Deprecations
~~~~~~~~~~~~

- Renamed vertical regridding schemes (:pull:`1429`) by :user:`schlunma`. Old regridding schemes are supported until v2.7.0. For details, see :ref:`Vertical interpolation schemes <Vertical interpolation schemes>`.

Documentation
~~~~~~~~~~~~~

- Remove duplicate entries in changelog (:pull:`1391`) by :user:`zklaus`
- Documentation on how to use HPC central installations (:pull:`1409`) by :user:`valeriupredoi`
- Correct brackets in preprocessor documentation for list of seasons (:pull:`1420`) by :user:`bouweandela`
- Add Python=3.10 to package info, update Circle CI auto install and documentation for Python=3.10 (:pull:`1432`) by :user:`valeriupredoi`
- Reverted unintentional change in `.zenodo.json` (:pull:`1452`) by :user:`schlunma`
- Synchronized config-user.yml with version from ESMValTool (:pull:`1453`) by :user:`schlunma`
- Solved issues in configuration files (:pull:`1457`) by :user:`schlunma`
- Add direct link to download conda lock file in the install documentation (:pull:`1462`) by :user:`valeriupredoi`
- CITATION.cff fix and automatic validation of citation metadata (:pull:`1467`) by :user:`valeriupredoi`
- Updated documentation on how to deprecate features (:pull:`1426`) by :user:`schlunma`
- Added reference hook to conda lock in documentation install section (:pull:`1473`) by :user:`valeriupredoi`
- Increased ESMValCore version to 2.5.0rc1 (:pull:`1477`) by :user:`schlunma`
- Added changelog for v2.5.0 release (:pull:`1476`) by :user:`schlunma`
- Increased ESMValCore version to 2.5.0rc2 (:pull:`1487`) by :user:`schlunma`
- Added some authors to citation and zenodo files (:pull:`1488`) by :user:`SarahAlidoost`
- Restored `scipy` intersphinx mapping (:pull:`1491`) by :user:`schlunma`
- Increased ESMValCore version to 2.5.0rc3 (:pull:`1504`) by :user:`schlunma`
- Fix download instructions for the MSWEP dataset (:pull:`1506`) by :user:`remi-kazeroni`
- Documentation updated for the new cmorizer framework (:pull:`1417`) by :user:`remi-kazeroni`
- Added tests for duplicates in changelog and removed duplicates (:pull:`1508`) by :user:`schlunma`
- Increased ESMValCore version to 2.5.0rc4 (:pull:`1519`) by :user:`schlunma`
- Add Github Actions Test badge in README (:pull:`1526`) by :user:`valeriupredoi`
- Increased ESMValCore version to 2.5.0rc5 (:pull:`1529`) by :user:`schlunma`
- Increased ESMValCore version to 2.5.0rc6 (:pull:`1532`) by :user:`schlunma`

Fixes for datasets
~~~~~~~~~~~~~~~~~~

- Added fix for AIRS v2.1 (obs4mips) (:pull:`1472`) by :user:`axel-lauer`

Preprocessor
~~~~~~~~~~~~

- Added bias preprocessor (:pull:`1406`) by :user:`schlunma`
- Improve error messages when a preprocessor is failing (:pull:`1408`) by :user:`schlunma`
- Added option to explicitly not use fx variables in preprocessors (:pull:`1416`) by :user:`schlunma`
- Add `extract_location` preprocessor to extract town, city, mountains etc - anything specifiable by a location (:pull:`1251`) by :user:`jvegreg`
- Add ensemble statistics preprocessor and 'groupby' option for multimodel (:pull:`673`) by :user:`sloosvel`
- Generic regridding preprocessor (:pull:`1448`) by :user:`zklaus`

Automatic testing
~~~~~~~~~~~~~~~~~

- Add `pandas` as dependency :panda_face: (:pull:`1402`) by :user:`valeriupredoi`
- Fixed tests for python 3.7 (:pull:`1410`) by :user:`schlunma`
- Remove accessing `.xml()` cube method from test (:pull:`1419`) by :user:`valeriupredoi`
- Remove flag to use pip 2020 solver from Github Action pip install command on OSX (:pull:`1357`) by :user:`valeriupredoi`
- Add Python=3.10 to Github Actions and switch to Python=3.10 for the Github Action that builds the PyPi package (:pull:`1430`) by :user:`valeriupredoi`
- Pin `flake8<4` to keep getting relevant error traces when tests fail with FLAKE8 issues (:pull:`1434`) by :user:`valeriupredoi`
- Implementing conda lock (:pull:`1164`) by :user:`valeriupredoi`
- Relocate `pytest-monitor` outputted database `.pymon` so `.pymon-journal` file should not be looked for by `pytest` (:pull:`1441`) by :user:`valeriupredoi`
- Switch to Mambaforge in Github Actions tests (:pull:`1438`) by :user:`valeriupredoi`
- Turn off conda lock file creation on any push on `main` branch from Github Action test (:pull:`1489`) by :user:`valeriupredoi`
- Add DRS path test for IPSLCM files (:pull:`1490`) by :user:`senesis`
- Add a test module that runs tests of `iris` I/O every time we notice serious bugs there (:pull:`1510`) by :user:`valeriupredoi`
- [Github Actions] Trigger Github Actions tests (`run-tests.yml` workflow) from a comment in a PR (:pull:`1520`) by :user:`valeriupredoi`
- Update Linux condalock file (various pull requests) github-actions[bot]

Installation
~~~~~~~~~~~~

- Move `nested-lookup` dependency to `environment.yml` to be installed from conda-forge instead of PyPi (:pull:`1481`) by :user:`valeriupredoi`
- Pinned `iris` (:pull:`1511`) by :user:`schlunma`
- Updated dependencies (:pull:`1521`) by :user:`schlunma`
- Pinned iris<3.2.0 (:pull:`1525`) by :user:`schlunma`

Improvements
~~~~~~~~~~~~

- Allow to load all files, first X years or last X years in an experiment (:pull:`1133`) by :user:`sloosvel`
- Filter tasks earlier (:pull:`1264`) by :user:`jvegreg`
- Added earlier validation for command line arguments (:pull:`1435`) by :user:`schlunma`
- Remove `profile_diagnostic` from diagnostic settings and increase test coverage of `_task.py` (:pull:`1404`) by :user:`valeriupredoi`
- Add `output2` to the `product` extra facet of CMIP5 data (:pull:`1514`) by :user:`remi-kazeroni`
- Speed up ESGF search (:pull:`1512`) by :user:`bouweandela`


.. _changelog-v2-4-0:

------

Highlights
~~~~~~~~~~

- ESMValCore now has the ability to automatically download missing data from ESGF. For details, see :ref:`Data Retrieval<data-retrieval>`.
- ESMValCore now also can resume an earlier run. This is useful to re-use expensive preprocessor results. For details, see :ref:`Running<running>`.

This release includes

Bug fixes
~~~~~~~~~

- Crop on the ID-selected region(s) and not on the whole shapefile (:pull:`1151`) by :user:`stefsmeets`
- Add 'comment' to list of removed attributes (:pull:`1244`) by :user:`Peter9192`
- Speed up multimodel statistics and fix bug in peak computation (:pull:`1301`) by :user:`bouweandela`
- No longer make plots of provenance (:pull:`1307`) by :user:`bouweandela`
- No longer embed provenance in output files (:pull:`1306`) by :user:`bouweandela`
- Removed automatic addition of areacello to obs4mips datasets (:pull:`1316`) by :user:`schlunma`
- Pin docutils <0.17 to fix bullet lists on readthedocs (:pull:`1320`) by :user:`zklaus`
- Fix obs4MIPs capitalization (:pull:`1328`) by :user:`bouweandela`
- Fix Python 3.7 tests (:pull:`1330`) by :user:`bouweandela`
- Handle fx variables in `extract_levels` and some time operations (:pull:`1269`) by :user:`sloosvel`
- Refactored mask regridding for irregular grids (fixes 772) (:pull:`865`) by :user:`zklaus`
- Fix `da.broadcast_to` call when the fx cube has different shape than target data cube (:pull:`1350`) by :user:`valeriupredoi`
- Add tests for _aggregate_time_fx (:pull:`1354`) by :user:`sloosvel`
- Fix extra facets (:pull:`1360`) by :user:`bouweandela`
- Pin pip!=21.3 to avoid pypa/pip10573 with editable installs (:pull:`1359`) by :user:`zklaus`
- Add a custom `date2num` function to deal with changes in cftime (:pull:`1373`) by :user:`zklaus`
- Removed custom version of `AtmosphereSigmaFactory` (:pull:`1382`) by :user:`schlunma`

Deprecations
~~~~~~~~~~~~

- Remove write_netcdf and write_plots from config-user.yml (:pull:`1300`) by :user:`bouweandela`

Documentation
~~~~~~~~~~~~~

- Add link to plot directory in index.html (:pull:`1256`) by :user:`stefsmeets`
- Work around issue with yapf not following PEP8 (:pull:`1277`) by :user:`bouweandela`
- Update the core development team (:pull:`1278`) by :user:`bouweandela`
- Update the documentation of the provenance interface (:pull:`1305`) by :user:`bouweandela`
- Update version number to first release candidate 2.4.0rc1 (:pull:`1363`) by :user:`zklaus`
- Update to new ESMValTool logo (:pull:`1374`) by :user:`zklaus`
- Update version number for third release candidate 2.4.0rc3 (:pull:`1384`) by :user:`zklaus`
- Update changelog for 2.4.0rc3 (:pull:`1385`) by :user:`zklaus`
- Update version number to final 2.4.0 release (:pull:`1389`) by :user:`zklaus`
- Update changelog for 2.4.0 (:pull:`1366`) by :user:`zklaus`

Fixes for datasets
~~~~~~~~~~~~~~~~~~

- Add fix for differing latitude coordinate between historical and ssp585 in MPI-ESM1-2-HR r2i1p1f1 (:pull:`1292`) by :user:`bouweandela`
- Add fixes for time and latitude coordinate of EC-Earth3 r3i1p1f1 (:pull:`1290`) by :user:`bouweandela`
- Apply latitude fix to all CCSM4 variables (:pull:`1295`) by :user:`bouweandela`
- Fix lat and lon bounds for FGOALS-g3 mrsos (:pull:`1289`) by :user:`thomascrocker`
- Add grid fix for tos in fgoals-f3-l (:pull:`1326`) by :user:`sloosvel`
- Add fix for CIESM pr (:pull:`1344`) by :user:`bouweandela`
- Fix DRS for IPSLCM : split attribute 'freq' into : 'out' and 'freq' (:pull:`1304`) by :user:`senesis`

CMOR standard
~~~~~~~~~~~~~

- Remove history attribute from coords (:pull:`1276`) by :user:`jvegreg`
- Increased flexibility of CMOR checks for datasets with generic alevel coordinates (:pull:`1032`) by :user:`schlunma`
- Automatically fix small deviations in vertical levels (:pull:`1177`) by :user:`bouweandela`
- Adding standard names to the custom tables of the `rlns` and `rsns` variables (:pull:`1386`) by :user:`remi-kazeroni`

Preprocessor
~~~~~~~~~~~~

- Implemented fully lazy climate_statistics (:pull:`1194`) by :user:`schlunma`
- Run the multimodel statistics preprocessor last (:pull:`1299`) by :user:`bouweandela`

Automatic testing
~~~~~~~~~~~~~~~~~

- Improving test coverage for _task.py (:pull:`514`) by :user:`valeriupredoi`
- Upload coverage to codecov (:pull:`1190`) by :user:`bouweandela`
- Improve codecov status checks (:pull:`1195`) by :user:`bouweandela`
- Fix curl install in CircleCI (:pull:`1228`) by :user:`jvegreg`
- Drop support for Python 3.6 (:pull:`1200`) by :user:`valeriupredoi`
- Allow more recent version of `scipy` (:pull:`1182`) by :user:`schlunma`
- Speed up conda build `conda_build` Circle test by using `mamba` solver via `boa` (and use it for Github Actions test too) (:pull:`1243`) by :user:`valeriupredoi`
- Fix numpy deprecation warnings (:pull:`1274`) by :user:`bouweandela`
- Unpin upper bound for iris (previously was at <3.0.4) (:pull:`1275`) by :user:`valeriupredoi`
- Modernize `conda_install` test on Circle CI by installing from conda-forge with Python 3.9 and change install instructions in documentation (:pull:`1280`) by :user:`valeriupredoi`
- Run a nightly Github Actions workflow to monitor tests memory per test (configurable for other metrics too) (:pull:`1284`) by :user:`valeriupredoi`
- Speed up tests of tasks (:pull:`1302`) by :user:`bouweandela`
- Fix upper case to lower case variables and functions for flake compliance in `tests/unit/preprocessor/_regrid/test_extract_levels.py` (:pull:`1347`) by :user:`valeriupredoi`
- Cleaned up a bit Github Actions workflows (:pull:`1345`) by :user:`valeriupredoi`
- Update circleci jobs: renaming tests to more descriptive names and removing conda build test (:pull:`1351`) by :user:`zklaus`
- Pin iris to latest `>=3.1.0` (:pull:`1341`) by :user:`valeriupredoi`

Installation
~~~~~~~~~~~~

- Pin esmpy to anything but 8.1.0 since that particular one changes the CPU affinity (:pull:`1310`) by :user:`valeriupredoi`

Improvements
~~~~~~~~~~~~

- Add a more friendly and useful message when using default config file (:pull:`1233`) by :user:`valeriupredoi`
- Replace os.walk by glob.glob in data finder (only look for data in the specified locations) (:pull:`1261`) by :user:`bouweandela`
- Machine-specific directories for auxiliary data in the `config-user.yml` file (:pull:`1268`) by :user:`remi-kazeroni`
- Add an option to download missing data from ESGF (:pull:`1217`) by :user:`bouweandela`
- Speed up provenance recording (:pull:`1327`) by :user:`bouweandela`
- Improve results web page (:pull:`1332`) by :user:`bouweandela`
- Move institutes from config-developer.yml to default extra facets config and add wildcard support for extra facets (:pull:`1259`) by :user:`bouweandela`
- Add support for re-using preprocessor output from previous runs (:pull:`1321`) by :user:`bouweandela`
- Log fewer messages to screen and hide stack trace for known recipe errors (:pull:`1296`) by :user:`bouweandela`
- Log ESMValCore and ESMValTool versions when running (:pull:`1263`) by :user:`jvegreg`
- Add "grid" as a tag to the output file template for CMIP6 (:pull:`1356`) by :user:`zklaus`
- Implemented ICON project to read native ICON model output (:pull:`1079`) by :user:`bsolino`


.. _changelog-v2-3-1:

------

This release includes

Bug fixes
~~~~~~~~~

- Update config-user.yml template with correct drs entries for CEDA-JASMIN (:pull:`1184`) by :user:`valeriupredoi`
- Enhancing MIROC5 fix for hfls and evspsbl (:pull:`1192`) by :user:`katjaweigel`
- Fix alignment of daily data with inconsistent calendars in multimodel statistics (:pull:`1212`) by :user:`Peter9192`
- Pin cf-units, remove github actions test for Python 3.6 and fix test_access1_0 and test_access1_3 to use cf-units for comparisons (:pull:`1197`) by :user:`valeriupredoi`
- Fixed search for fx files when no ``mip`` is given (:pull:`1216`) by :user:`schlunma`
- Make sure climate statistics always returns original dtype (:pull:`1237`) by :user:`zklaus`
- Bugfix for regional regridding when non-integer range is passed (:pull:`1231`) by :user:`stefsmeets`
- Make sure area_statistics preprocessor always returns original dtype (:pull:`1239`) by :user:`zklaus`
- Add "." (dot) as allowed separation character for the time range group (:pull:`1248`) by :user:`zklaus`

Documentation
~~~~~~~~~~~~~

- Add a link to the instructions to use pre-installed versions on HPC clusters (:pull:`1186`) by :user:`remi-kazeroni`
- Bugfix release: set version to 2.3.1 (:pull:`1253`) by :user:`zklaus`

Fixes for datasets
~~~~~~~~~~~~~~~~~~

- Set circular attribute in MCM-UA-1-0 fix (:pull:`1178`) by :user:`sloosvel`
- Fixed time coordinate of MIROC-ESM (:pull:`1188`) by :user:`schlunma`

Preprocessor
~~~~~~~~~~~~

- Filter warnings about collapsing multi-model dimension in multimodel statistics preprocessor function (:pull:`1215`) by :user:`bouweandela`
- Remove fx variables before computing multimodel statistics (:pull:`1220`) by :user:`sloosvel`

Installation
~~~~~~~~~~~~

- Pin lower bound for iris to 3.0.2 (:pull:`1206`) by :user:`valeriupredoi`
- Pin `iris<3.0.4` to ensure we still (sort of) support Python 3.6 (:pull:`1252`) by :user:`valeriupredoi`

Improvements
~~~~~~~~~~~~

- Add test to verify behaviour for scalar height coord for tas in multi-model (:pull:`1209`) by :user:`Peter9192`
- Sort missing years in "No input data available for years" message (:pull:`1225`) by :user:`ledm`


.. _changelog-v2-3-0:

------

This release includes

Bug fixes
~~~~~~~~~

- Extend preprocessor multi_model_statistics to handle data with "altitude" coordinate (:pull:`1010`) by :user:`axel-lauer`
- Remove scripts included with CMOR tables (:pull:`1011`) by :user:`bouweandela`
- Avoid side effects in extract_season (:pull:`1019`) by :user:`jvegreg`
- Use nearest scheme to avoid interpolation errors with masked data in regression test (:pull:`1021`) by :user:`stefsmeets`
- Move _get_time_bounds from preprocessor._time to cmor.check to avoid circular import with cmor module (:pull:`1037`) by :user:`valeriupredoi`
- Fix test that makes conda build fail (:pull:`1046`) by :user:`valeriupredoi`
- Fix 'positive' attribute for rsns/rlns variables (:pull:`1051`) by :user:`lukasbrunner`
- Added preprocessor mask_multimodel (:pull:`767`) by :user:`schlunma`
- Fix bug when fixing bounds after fixing longitude values (:pull:`1057`) by :user:`sloosvel`
- Run conda build parallel AND sequential tests (:pull:`1065`) by :user:`valeriupredoi`
- Add key to id_prop (:pull:`1071`) by :user:`lukasbrunner`
- Fix bounds after reversing coordinate values (:pull:`1061`) by :user:`sloosvel`
- Fixed --skip-nonexistent option (:pull:`1093`) by :user:`schlunma`
- Do not consider CMIP5 variable sit to be the same as sithick from CMIP6 (:pull:`1033`) by :user:`bouweandela`
- Improve finding date range in filenames (enforces separators) (:pull:`1145`) by :user:`senesis`
- Review fx handling (:pull:`1147`) by :user:`sloosvel`
- Fix lru cache decorator with explicit call to method (:pull:`1172`) by :user:`valeriupredoi`
- Update _volume.py (:pull:`1174`) by :user:`ledm`

Deprecations
~~~~~~~~~~~~



Documentation
~~~~~~~~~~~~~

- Final changelog for 2.3.0 (:pull:`1163`) by :user:`zklaus`
- Set version to 2.3.0 (:pull:`1162`) by :user:`zklaus`
- Fix documentation build (:pull:`1006`) by :user:`bouweandela`
- Add labels required for linking from ESMValTool docs (:pull:`1038`) by :user:`bouweandela`
- Update contribution guidelines (:pull:`1047`) by :user:`bouweandela`
- Fix basestring references in documentation (:pull:`1106`) by :user:`jvegreg`
- Updated references master to main (:pull:`1132`) by :user:`axel-lauer`
- Add instructions how to use the central installation at DKRZ-Mistral (:pull:`1155`) by :user:`remi-kazeroni`

Fixes for datasets
~~~~~~~~~~~~~~~~~~

- Added fixes for various CMIP5 datasets, variable cl (3-dim cloud fraction) (:pull:`1017`) by :user:`axel-lauer`
- Added fixes for hybrid level coordinates of CESM2 models (:pull:`882`) by :user:`schlunma`
- Extending LWP fix for CMIP6 models (:pull:`1049`) by :user:`axel-lauer`
- Add fixes for the net & up radiation variables from ERA5 (:pull:`1052`) by :user:`lukasbrunner`
- Add derived variable rsus (:pull:`1053`) by :user:`lukasbrunner`
- Supported `mip`-level fixes (:pull:`1095`) by :user:`schlunma`
- Fix erroneous use of `grid_latitude` and `grid_longitude` and cleaned ocean grid fixes (:pull:`1092`) by :user:`schlunma`
- Fix for pr of miroc5 (:pull:`1110`) by :user:`remi-kazeroni`
- Ocean depth fix for cnrm_esm2_1, gfdl_esm4, ipsl_cm6a_lr datasets + mcm_ua_1_0 (:pull:`1098`) by :user:`tomaslovato`
- Fix for uas variable of the MCM_UA_1_0 dataset (:pull:`1102`) by :user:`remi-kazeroni`
- Fixes for sos and siconc of BCC models (:pull:`1090`) by :user:`remi-kazeroni`
- Run fgco2 fix for all CESM2 models (:pull:`1108`) by :user:`LisaBock`
- Fixes for the siconc variable of CMIP6 models (:pull:`1105`) by :user:`remi-kazeroni`
- Fix wrong sign for land surface flux (:pull:`1113`) by :user:`LisaBock`
- Fix for pr of EC_EARTH (:pull:`1116`) by :user:`remi-kazeroni`

CMOR standard
~~~~~~~~~~~~~

- Format cmor related files (:pull:`976`) by :user:`jvegreg`
- Check presence of time bounds and guess them if needed (:pull:`849`) by :user:`sloosvel`
- Add custom variable "tasaga" (:pull:`1118`) by :user:`LisaBock`
- Find files for CMIP6 DCPP startdates (:pull:`771`) by :user:`sloosvel`

Preprocessor
~~~~~~~~~~~~

- Update tests for multimodel statistics preprocessor (:pull:`1023`) by :user:`stefsmeets`
- Raise in extract_season and extract_month if result is None (:pull:`1041`) by :user:`jvegreg`
- Allow selection of shapes in extract_shape (:pull:`764`) by :user:`jvegreg`
- Add option for regional regridding to regrid preprocessor (:pull:`1034`) by :user:`stefsmeets`
- Load fx variables as cube cell measures / ancillary variables (:pull:`999`) by :user:`sloosvel`
- Check horizontal grid before regridding (:pull:`507`) by :user:`BenMGeo`
- Clip irregular grids (:pull:`245`) by :user:`bouweandela`
- Use native iris functions in multi-model statistics (:pull:`1150`) by :user:`Peter9192`

Notebook API (experimental)
~~~~~~~~~~~~~~~~~~~~~~~~~~~



Automatic testing
~~~~~~~~~~~~~~~~~

- Report coverage for tests that run on any pull request (:pull:`994`) by :user:`bouweandela`
- Install ESMValTool sample data from PyPI (:pull:`998`) by :user:`jvegreg`
- Fix tests for multi-processing with spawn method (i.e. macOSX with Python>3.8) (:pull:`1003`) by :user:`bvreede`
- Switch to running the Github Action test workflow every 3 hours in single thread mode to observe if Segmentation Faults occur (:pull:`1022`) by :user:`valeriupredoi`
- Revert to original Github Actions test workflow removing the 3-hourly test run with -n 1 (:pull:`1025`) by :user:`valeriupredoi`
- Avoid stale cache for multimodel statistics regression tests (:pull:`1030`) by :user:`bouweandela`
- Add newer Python versions in OSX to Github Actions (:pull:`1035`) by :user:`bvreede`
- Add tests for type annotations with mypy (:pull:`1042`) by :user:`stefsmeets`
- Run problematic cmor tests sequentially to avoid segmentation faults on CircleCI (:pull:`1064`) by :user:`valeriupredoi`
- Test installation of esmvalcore from conda-forge (:pull:`1075`) by :user:`valeriupredoi`
- Added additional test cases for integration tests of data_finder.py (:pull:`1087`) by :user:`schlunma`
- Pin cf-units and fix tests (cf-units>=2.1.5) (:pull:`1140`) by :user:`valeriupredoi`
- Fix failing CircleCI tests (:pull:`1167`) by :user:`bouweandela`
- Fix test failing due to fx files chosen differently on different OS's (:pull:`1169`) by :user:`valeriupredoi`
- Compare datetimes instead of strings in _fixes/cmip5/test_access1_X.py (:pull:`1173`) by :user:`valeriupredoi`
- Pin Python to 3.9 in environment.yml on CircleCI and skip mypy tests in conda build (:pull:`1176`) by :user:`bouweandela`

Installation
~~~~~~~~~~~~

- Update yamale to version 3 (:pull:`1059`) by :user:`zklaus`

Improvements
~~~~~~~~~~~~

- Refactor diagnostics / tags management (:pull:`939`) by :user:`stefsmeets`
- Support multiple paths in input_dir (:pull:`1000`) by :user:`jvegreg`
- Generate HTML report with recipe output (:pull:`991`) by :user:`stefsmeets`
- Add timeout to requests.get in _citation.py (:pull:`1091`) by :user:`SarahAlidoost`
- Add SYNDA drs for CMIP5 and CMIP6 (closes 582) (:pull:`583`) by :user:`zklaus`
- Add basic support for variable mappings (:pull:`1124`) by :user:`zklaus`
- Handle IPSL-CM6 (:pull:`1153`) by :user:`senesis`


.. _changelog-v2-2-0: