Datacube

RC release to facilitate development in downstream packages using the new transaction API.

Most notable new feature is the new database transaction API, as discussed in [ODC-EP07 Database Transaction API](EP07-Database-transaction-API). API Example:


with dc.index.transaction() as trans:
Archive old datasets and add new ones in single transaction
dc.index.datasets.archive([old_ds1.id, old_ds2.id])
dc.index.datasets.add(ds1)
dc.index.datasets.add(ds2)

If execution gets to here, the transaction is committed.
If an exception was raised by any of the above methods, the transaction is rolled back.


Also includes an update of the main test docker build to Ubuntu 22.04 and Python 3.10, and significant progress on the new postgis index driver.

Please note that the postgis index driver is still flagged as "experimental" and is missing several key features.

- Cleanup mypy typechecking compliance. (1266)
- When dataset add operations fail due to lineage issues, the produced error message now clearly indicates that
the problem was due to lineage issues. (1260)
- Added support for group-by financial years to virtual products. (1257, 1261)
- Remove reference to `rasterio.path`. (1255)
- Cleaner separation of (experimental) postgis and (stable) postgres drivers, and suppress SQLAlchemy cache warnings. (1254)
- Prevent Shapely deprecation warning. (1253)
- Fix `DATACUBE_DB_URL` parsing to understand syntax like: `postgresql:///datacube?host=/var/run/postgresql` (1256)
- Clearer error message when local metadata file does not exist. (1252)
- Address upstream security alerts and update upstream library versions. (1250)
- Clone ``postgres`` index driver as ``postgis``, and flag as experimental. (1248)
- Implement a local non-persistent in-memory index driver, with maximal backwards-compatibility
with default postgres index driver. Doesn't work with CLI interface, as every invocation
will receive a new, empty index, but useful for testing and small scale proof-of-concept
work. (1247)
- Performance and correctness fixes backported from ``odc-geo``. (1242)
- Deprecate use of the celery executor. Update numpy pin in rtd-requirements.txt to suppress
Dependabot warnings. (1239)
- Implement a minimal "null" index driver that provides an always-empty index. Mainly intended
to validate the recent abstraction work around the index driver layer, but may be useful
for some testing scenarios, and ODC use cases that do not require an index. (1236)
- Regularise some minor API inconsistencies and restore redis-server to Docker image. (1234)
- Move (default) postgres driver-specific files from `datacube.index` to `datacube.index.postgres`.
`datacube.index.Index` is now an alias for the abstract base class index interface definition
rather than postgres driver-specific implementation of that interface. (1231)
- Update numpy and netcdf4 version in docker build (1229)
rather than postgres driver-specific implementation of that interface. (1227)
- Migrate test docker image from `datacube/geobase` to `osgeo/gdal`. (1233)
- Separate index driver interface definition from default index driver implementation. (1226)
- Prefer WKT over EPSG when guessing CRS strings. (1223, 1262)
- Updates to documentation. (1208, 1212, 1215, 1218, 1240, 1244)
- Tweak to segmented in geometry to suppress Shapely warning. (1207)
- Fix to ensure ``skip_broken_datasets`` is correctly propagated in virtual products (1259)
- Deprecate `Rename`, `Select` and `ToFloat` built-in transforms in virtual products (1263)

Includes contributions from whatnick, alexgleith, maawoo, jeremyh, iamtekson, alfredoahds, SpacemanPaul, kirill888, robbitbt, tebadi, uchchwhash, and mpaget.

Acknowledgements to the Open Datacube Steering Council and all supporting organisations, including Geoscience Australia, Digital Earth Africa, CSIRO, Frontier SI and Aerometrex.

- Added `dataset purge` command for hard deletion of archived datasets 1199
- Trivial fixes to CLI help output 1197
- Fix to enable searching for multiple products 1201

* Fix unguarded dependencies on boto libraries 1174 1172
* Various documentation fixes 1175
* Address import problems on Windows due to use of Unix only functions 1176
* Address `numpy.bool` deprecation warnings 1184

=======================

* Removed example and contributed notebooks from the repository. Better [notebook examples](https://github.com/GeoscienceAustralia/dea-notebooks/) exist
* Removed datacube_apps, as these are not used and not maintained
* Add `cloud_cover` to EO3 metadata
* Add `erosion` functionality to Virtual products' `ApplyMask` to supplement existing `dilation` functionality 1049
* Fix numeric precision issues in `compute_reproject_roi` when pixel size is small 1047
* Follow up fix to 1047 to round scale to nearest integer if very close
* Add support for 3D Datasets 1099
* New feature: search by URI from the command line `datacube dataset uri-search`
* Added new "license" and "description" properties to `DatasetType` to enable easier access to product information 1143 1144
* Revised the `Datacube.list_products` function to produce a simpler and more useful product list table 1145
* Refactor docs, making them more up to date and simpler 1137 1128
* Add new `dataset_predicate` param to `dc.load` and `dc.find_datasets` for more flexible temporal filtering (e.g. loading data for non-contiguous time ranges such as specific months or seasons over multiple years) 1148 1156
* Fix to `GroupBy` to ensure output output axes are correctly labelled when sorting observations using `sort_key` 1157
* `GroupBy` is now its own class to allow easier custom grouping and sorting of data 1157
* add support for IAM authentication for RDS databases in AWS 1168