Datacube

Major changes in this release:

1) The attempted fix to the projection bug in the previous release turned out to be inadequate, this release contains a fix for the bug fix.
2) The behaviour of the `archive-less-mature` option on dataset-add has been enhanced (default time leniency increased to +/-500ms, with the option for the user to set their own leniency threshold.
3) Several old and poorly maintained APIs have been formally deprecated in this release, and will be removed from datacube-core in the `1.9.0` release: ingestion (which the documentation has recommended against using for many years), executors and grid workflow. If you still use any of these APIs, you can continue to use them in this release (and will still be able to use them in all future `1.8.x` releases) by simply ignoring the deprecation warnings. If you have workflows that still depend on these APIs, please raise your requirements on the ODC Slack community so we can help you identify alternative approaches. There is no planned release date for v1.9.0 at this stage.

Full list of changes in this release:

- Second attempt to address unexpected handling of image aspect ratios in rasterio and GDAL. (1457)
- Fix broken PyPI publishing Github action (1454)
- Documentation improvements (1455)
- Increase default maturity leniency to +-500ms (1458)
- Add option to specify maturity timedelta when using `--archive-less-mature` option (1460)
- Mark executors as deprecated (1461)
- Mark ingestion as deprecated (1463)
- Replace deprecated `pkg_resources` with `importlib.resources` and `importlib.metadata` (1466)
- Update whats_new.rst for release (1467)

Includes code contributions from SpacemanPaul and Ariana-B.

With thanks to Kirill888, robbibt and snowman2 for their help in identifying and finding a viable fix for the projection bug and to Geoscience Australia for their continuing support of ODC development and maintenance.

Key improvements this release:

1) Fix for the upstream reprojection bug identified by robbibt
2) Implementation of "archive less mature" functionality in `datacube dataset add`.

Full list of changes:

- Fix broken Github action workflows (1425, 1427, 1433)
- Setup Dependabot, and Dependabot-generated updates (1416, 1420, 1423, 1428, 1436, 1447)
- Documentation fixes (1417, 1418, 1430)
- `datacube dataset` cli commands print error message if missing argument (1437)
- Add pre-commit hook to verify license headers (1438)
- Support open-ended date ranges in `datacube dataset search`, `dc.load`, and `dc.find_datasets` (1439, 1443)
- Pass Y and Y Scale factors through to rasterio.warp.reproject, to eliminate projection bug affecting
non-square Areas Of Interest (See Issue 1448) (1450)
- Add `archive_less_mature` option to `datacube dataset add` and `datacube dataset update` (1451)
- Allow for +-1ms leniency in finding other maturity versions of a dataset (1452)
- Update whats_new.rst for release (1453)

With contributions from SpacemanPaul, Ariana-B, omad, and MartinPontius

Special thanks to Kirill888 for his assistance with resolving 1448 and thanks to all supporting organisations, especially Geoscience Australia.

The previous two releases had a serious bug in the new Transaction API that resulted in parallel writes to the database being highly likely to fail. This release is mostly to provide a fix for this issue.

Full list of changes:

- Rename Geometry `type` attribute to `geom_type`, to align with Shapely 2.0 (1402)
- Remove some deprecated SQLAlchemy usages (1403, 1407)
- Fix RTD docs build (1399)
- Minor Documentation fixes (1409, 1413)
- Bug-fix and code cleanup in virtual products (1410)
- Reduce transaction isolation level to improve database write concurrency (1414)
- Update dependency versions and whats_new.rst for release (1415)

Includes contributions from SpacemanPaul, emmaai, Ariana-B, omad, and first-time contributor charalamm.

Special thanks to alexgleith for reporting the Transaction API bug and working with us to test the fix.

Administrative release to address two technical issues in release 1.8.10:

1) 1.8.10 attempted to improve compatibility with recent versions of xarray, however further work was needed to fully achieve that compatibility.
2) 1.8.10 was released with dependency discrepancies between `setup.py` (pip), `conda-environment.yml` (conda) and `constrains.in` (Docker).

Full list of changes:

- Simplify Github actions (1393)
- Update conda create environment README (1394)
- Update conda environment file and add notes to release process to ensure pip and conda
dependencies are in sync and up-to-date. (1395)
- Update docker constraints (1396)
- Compatible with the changes w.r.t. `MultiIndex` and `coord/dims` introduced since `xarray>2022.3.0` (1397)
- Final sync of conda/pip dependencies and update release notes. (1398)

Includes contributions from SpacemanPaul, pindge, Ariana-B, emmaai and omad.

Bug fixes, enhancements, and updates to work with newer versions of upstream libraries.

Notes

1. The new APIs for bulk-reads, bulk-writes and index cloning should be considered unstable and may change in subsequent releases.
2. Recent refactoring in the XArray library has lead to changes in behaviour that affect some ODC operations and are unlikely to be addressed by the XArray team. This release includes changes in the way the ODC works with XArray to circumvent these issues. If you experience Xarray issues with this ODC release, please raise an issue on Github and we will try to address them before the next release.

Full list of changes:

- Add `grid_spec` to `list_products` (1357)
- Add database relationship diagram to doc (1350)
- Maintain search field index tables, and use them for dataset queries (1360)
- Change Github lint action to use ``conda`` and remove ``flake8`` from action (1361)
- Fix database relationship diagram instruction for docker (1362)
- Document ``group_by`` for ``dataset.load`` (1364)
- Add search_by_metadata facility for products (1366)
- Postgis driver cleanup - remove faux support for lineage (1368)
- Add support for nested database transactions (1369)
- Fix Github doc lint action (1370)
- Tighten EO3 enforcement in postgis driver, refactor tests, and rename Dataset.type to Dataset.product
(with type alias for compatibility) (1372)
- Fix deprecation message due to distutils Version classes (1375)
- Postgresql drivers cleanup - consolidate split_uri into utils and removed unused constants (1378)
- Postgresql drivers cleanup - Handle NaNs in search fields and allow caching in sanitise_extent (1379)
- Fix example product yaml documentation (1384)
- Bulk read/write API methods and fast whole-index cloning. Cloning does NOT include lineage information yet,
and new API methods may be subject to change. (1381)
- Documentation update. (1385)
- Clean up datetime functions (1387)
- Dependency updates (1388, 1391)
- Upgrades for compatibility with newer versions of Shapely and Xarray. (1389)
- Finalise release notes for 1.8.10 release (1392)

Includes contributions from: pindge, omad, Ariana-B, sanjeev0210 and SpacemanPaul

Most notable changes:

1. `patch_url` argument to `dc.load()` and `dc.load_data()` (introduced in v1.8.8) is now also supported for Dask loading.
2. Fixed a day-zero bug affecting search over range-type search fields where the target and search field value is of zero-width.
3. Performance improvements to CRS geometry class.
4. Numerous improvements to documentation and github actions.

Full list of changes:

* Performance improvements to CRS geometry class (1322)
* Extend `patch_url` argument to `dc.load()` and `dc.load_data()` to Dask loading. (1323)
* Add `sphinx.ext.autoselectionlabel` extension to readthedoc conf to support `:ref:` command (1325)
* Add `pyspellcheck` for `.rst` documentation files and fix typos (1327)
* Add `rst` documentation lint github action and apply best practices (1328)
* Follow PEP561_ to make type hints available to other packages (1331)
* Updated GitHub actions config to remove deprecated `set-output` (1333)
* Add what's new page link to menu and general doc fixes (1335)
* Add `search_fields` to required for metadata type schema and update doc (1339)
* Fix typo and update metadata documentation (1340)
* Add readthedoc preview github action (1344)
* Update `nodata` in readthedoc for products page (1347)
* Add `eo-datasets` to extensions & related software doc page (1349)
* Fix bug affecting searches against range types of zero width (1352)
* Add 1.8.9 release date and missing PR to `whats_news.rst` (1353)

Includes contributions from SpacemanPaul, omad, pindge, snowman2.

With thanks and appreciation to all contributors, users and supporting organisations, expecially Geoscience Australia.