Openghg

Added

- Multiple object stores are now supported. Any number of stores may be accessed and read from and written to - [PR 664](https://github.com/openghg/openghg/pull/664)
- Added `standardise_column()` wrapper function - [PR 569](https://github.com/openghg/openghg/pull/643)
- The 'height_name' definition from the [openghg/supplementary_data repository](https://github.com/openghg/supplementary_data) for each site can now be accessed, used and interpreted - [PR #648](https://github.com/openghg/openghg/pull/648)
- Allow metadata within metastore to be updated for existing data sources for the latest version, start and end dates of the data - [PR 652](https://github.com/openghg/openghg/pull/652)
- Allow mismatches between values in metadata and data attributes to be updated to either match to the metadata or attribute collated details - [PR 682](https://github.com/openghg/openghg/pull/682)

Changed

- Configuration file format has been updated to support multiple object stores, note directing users to upgrade has been added
- The `openghg --quickstart` functionality has been updated to allow multiple objects stores to be added and migrate users from the previous version of the configuration file
- Synchronise metadata within metastore to align with data sources when updating (including latest version, start and end dates of the data) - [PR 652](https://github.com/openghg/openghg/pull/652) and [PR #664](https://github.com/openghg/openghg/pull/664)
- The name of the `DataHandler` class has been changed to `DataManager` to better reflect its function.

Removed

- Reading multi-site AQMesh data is no longer possible. This may be reintroduced if required.

Fixed

- Fix for the sampling period of data files being read incorrectly - [PR 584](https://github.com/openghg/openghg/pull/584)
- Fix for overlapping dateranges being created when adding new data to a Datasource. This introduced errors when keys were being removed and updated - [PR 570](https://github.com/openghg/openghg/pull/570)
- Incorrect data being retrieved by the ICOS retrieval function - [PR 611](https://github.com/openghg/openghg/pull/611)
- Error raised on attempt to delete object store after it wasn't created by some tests - [PR 626](https://github.com/openghg/openghg/pull/626)
- Very small (nanosecond) changes in period between measurements resulting in error due to `pandas.infer_period` not being able to return a period - [PR 634](https://github.com/openghg/openghg/pull/634)
- Processing of ObsPack resulted in errors due to limited metadata read and data overwrite, temporary fix in place for now - [PR 642](https://github.com/openghg/openghg/pull/642)

Changed

- Mock for `openghg_defs` data to remove external dependency for tests - [PR 582](https://github.com/openghg/openghg/pull/582)
- Removed use of environment variable for test store, moved to mock - [PR 580](https://github.com/openghg/openghg/pull/580)
- Temporary pinning of pandas < 2.0 due to changes that introduced errors - [PR 619](https://github.com/openghg/openghg/pull/619)
- `ObsData.plot_timeseries` now uses `openghg.plotting.plot_timeseries` to avoid duplication in efforts/code - [PR 624](https://github.com/openghg/openghg/pull/624)
- `openghg.util._user.get_user_config_path` now creates `openghg.conf` in `~/.openghg` - [PR 690](https://github.com/openghg/openghg/pull/690)

Added

- New tutorial on changing object store path using command line
- New tutorial on adding data from EDGAR database
- Ability to use separate
- New keywords to allow metadata for emissions data to be more specific
- New command-line tool to setup user configuration file using `openghg --quickstart`
- New installation guide for users and developers
- Added check for 0-dimension time coordinates in some NetCDFs

Fixed

- Fix for retrieval of different ICOS datasets
- Fix for user configuration file not being written out correctly

Removed

- Supplementary site and species data moved to separate [supplementary_data](https://github.com/openghg/supplementary_data) repository
- Unused test data files

Changed

- Print statements changed to logging
- Updated version of mypy used to 0.991
- Converted all tutorial notebooks to restructured text files

Added

- A new `DataHandler` class for modification and deletion of metadata and added a matching tutorial.
- Move to a new user configuration file.
- A new `ObsColumn` class for handing satellite data

Fixed

- Fixes for search parameters such as inlet height, sampling height
- Removed `date` parameter from Flux data to improve searchability
- Inlet and height are now aliases for each other for footprints and obs_surface
- Check for tuple of data and precisions file on processing of GCWERKS data

Changed

- Documentation overhaul to ensure all functions are visible in either the user or developer documentation
- The `OPENGHG_PATH` environment variable check has been deprecated in favour of the user config file.

Fixed

- Removed old `footprints_data_merge` workflow which is superceded by `ModelScenario`.
- Tidied and updated tutorial notebooks
- Updated the conda build recipe

Removed

- Removed unused `jobs` submodule

Added

- Full ICOS and CEDA archive pulling capabilities in the cloud and locally.
- Adds logging to logfile, stored at `~/openghg_log` when run locally.
- Improved schema checking for different file formats, including footprint files.

Fixed

- Fixes the `clean_string` function of `openghg.util` to allow dashes as these are commonly used in species names.
- Improves local routing functionality within cloud functions

Changed

- OpenGHG now only supports Python >= 3.8