Pycontrails

Support "dry advection" simulation.

Features

- Add new `DryAdvection` model to simulate sediment-free advection of an aircraft's exhaust plume. This model is experimental and may change in future releases. By default, the current implementation simulates plume geometry as a cylinder with an elliptical cross section (the same geometry assumed in CoCiP). Wind shear perturbs the ellipse azimuth, width, and depth over the plume evolution. The `DryAdvection` model may also be used to simulate advection without wind-shear effects by setting the model parameters `azimuth`, `width`, and `depth` to None.
- Add new [Dry Advection example notebook](https://py.contrails.org/examples/advection.html) highlighting the new `DryAdvection` model and comparing it to the `Cocip` model.
- Add optional `fill_value` parameter to `VectorDataset.sum`.

Fixes

- (80) Fix unit error in `wake_vortex.turbulent_kinetic_energy_dissipation_rate`. This fix affects the estimate of wake vortex max downward displacement and slightly changes `Cocip` predictions.
- Change the implementation of `Flight.resample_and_fill` so that lat-lon interpolation is linear in time. Previously, the timestamp associated to a waypoint was floored according to the resampling frequency without updating the position accordingly. This caused errors in segment calculations (e.g., true airspeed).

Internals

- Add optional `keep_original_index` parameter to the `Flight.resample_and_fill` method. If `True`, the time original index is preserved in the output in addition to the new time index obtained by resampling.
- Improve docstrings in `wake_vortex` module
- Rename `wake_vortex` functions to remove `get_` prefix at the start of each function name.
- Add pytest command line parameter `--regenerate-results` to regenerate static test fixture results. Automate in make recipe `make pytest-regenerate-results`.
- Update handling of `GeoVectorDataset.required_keys` the `GeoVectorDataset` constructor. Add class variable `GeoVectorDataset.vertical_keys` for handing the vertical dimension.
- Rename `CocipParam.max_contrail_depth` -> `CocipParam.max_depth`.
- Add `units.dt_to_seconds` function to convert `np.timedelta64` to `float` seconds.
- Rename `thermo.p_dz` -> `thermo.pressure_dz`.

Add experimental support for simulating radiative effects due to contrail-contrail overlap.

Features

- Support simulating contrail contrail overlap when running the `Cocip` model with a `Fleet` source. The `contrail_contrail_overlapping` and `dz_overlap_m` parameters govern the overlap calculation. This mode of calculation is still experimental and may change in future releases.
- Rewrite the `pycontrails.models.cocip.output` modules into a single `pycontrails.cocip.output_formats` module. The new module supports flight waypoint summary statistics, contrail waypoints summary statistics, gridded outputs, and time-slice outputs.
- Add new `GeoVectorDataset.to_lon_lat_grid` method. This method can be thought of as a partial inverse to the `MetDataset.to_vector` method. The current implementation is brittle and may be revised in a future release.

Fixes

- Extend `Models.set_source_met` to allow `interpolation_q_method="cubic-spline"` when working with `MetDataset` source (ie, so-called gridded models). Previously a `NotImplementedError` was raised.
- Ensure the `Flight.copy` implementation works with `Fleet` instances as well.
- Avoid looping over `keys` twice in `VectorDataset.broadcast_attrs`. This is a slight performance enhancement.
- Fix `Fleet` signature for compatibility with `Flight`.
- Fix a few hard-coded assumptions in broadcasting aircraft performance and emissions when running `Cocip` with a `Fleet` source. The previous implementation did not consider the possibility of aircraft performance variables on `Flight.data` and `Flight.attrs` separately.

Internals

- Add optional `raise_error` parameter to the `VectorDataset.broadcast_attrs` method.
- Update `Fleet` internals.

Fixes

- Narrow type hints on the ABC `AircraftPerformance` model. The `AircraftPerformance.eval` method requires a `Flight` object for the `source` parameter.
- In `PSFlight.eval`, explicitly set any aircraft performance data at waypoints with zero true airspeed to `np.nan`. This avoids numpy `RuntimeWarning`s without affecting the results.
- Fix corner-case in the `polygon.buffer_and_clean` function in which the polygon created by buffering the `opencv` contour is not valid. Now a second attempt to buffer the polygon is made with a smaller buffer distance.
- Ignore RuntimeError raised in `scipy.optimize.newton` if the maximum number of iterations is reached before convergence. This is a workaround for occasional false positive convergence warnings. The pycontrails use-case may be related to [this GitHub issue](https://github.com/scipy/scipy/issues/8904).
- Update the `Models.__init__` warnings when `humidity_scaling` is not provided. The previous warning provided an outdated code example.
- Ensure the `interpolation_q_method` used in a parent model is passed into the `humidity_scaling` child model in the `Models.__init__` method. If the two `interpolation_q_method` values are different, a warning is issued. This could be extended to other model parameters in the future.

Features

- Enable `ExponentialBoostLatitudeCorrectionHumidityScaling` humidity scaling for the model parameter `interpolation_q_method="cubic_spline"`.
- Add [GFS notebook](https://py.contrails.org/examples/GFS.html) example.

Breaking changes

- Remove `ExponentialBoostLatitudeCorrectionHumidityScalingParams`. These parameters are now hard-coded in the `ExponentialBoostLatitudeCorrectionHumidityScaling` model.

Breaking changes

- By default, call `xr.open_mfdataset` with `lock=False` in the `MetDataSource.open_dataset` method. This helps alleviate a `dask` threading issue similar to [this GitHub issue](https://github.com/pydata/xarray/issues/4406).

Fixes

- Support `MetDataset` source in the `HistogramMatching` humidity scaling model. Previously only `GeoVectorDataset` sources were explicitly supported.
- Replace `np.gradient` with `dask.array.gradient` in the `tau_cirrus` module. This ensures that the computation is done lazily for dask-backed arrays.
- Round to 6 digits in the `polygon.determine_buffer` function. This avoid unnecessary warnings for rounding errors.
- Fix type hint for `opencv-python` 4.8.0.74.

Internals

- Take more care with `float` and `int` types in the `contrail_properties` module. Prefer `np.clip` to `np.where` or `np.maximum` for clipping values.
- Support `air_temperature` in `CocipGrid` verbose formation outputs.
- Remove `pytest-timeout` dev dependency.

Support for the [Poll-Schumann aircraft performance model](https://doi.org/10.1017/aer.2020.62).

Features

- Implement a basic working version of the Poll-Schumann (PS) aircraft performance model. This is experimental and may undergo revision in future releases. The PS Model currently supports the following 53 aircraft types:
- A30B
- A306
- A310
- A313
- A318
- A319
- A320
- A321
- A332
- A333
- A342
- A343
- A345
- A346
- A359
- A388
- B712
- B732
- B733
- B734
- B735
- B736
- B737
- B738
- B739
- B742
- B743
- B744
- B748
- B752
- B753
- B762
- B763
- B764
- B77L
- B772
- B77W
- B773
- B788
- B789
- E135
- E145
- E170
- E195
- MD82
- MD83
- GLF5
- CRJ9
- DC93
- RJ1H
- B722
- A20N
- A21N

The "gridded" version of this model is not yet implemented. This will be added in a future release.
- Improve the runtime of instantiating the `Emissions` model by a factor of 10-15x. This translates to a time savings of several hundred milliseconds on modern hardware. This improvement is achieved by more efficient parsing of the underlying static data and by deferring the construction of the interpolation artifacts until they are needed.
- Automatically use a default engine type from the aircraft type in the `Emissions` model if an `engine_uid` parameter is not included on the `source`. This itself is configurable via the `use_default_engine_uid` parameter on the `Emissions` model. The default mappings from aircraft types to engines is included in `pycontrails/models/emissions/static/default-engine-uids.csv`.

Breaking changes

- Remove the `Aircraft` dataclass from `Flight` instantiation. Any code previously using this should instead directly pass additional `attrs` to the `Flight` constructor.
- The `load_factor` is now required in `AircraftPerformance` models. The global `DEFAULT_LOAD_FACTOR` constant in `pycontrails.models.aircraft_performance` provides a reasonable default. This is currently set to 0.7.
- Use a `takeoff_mass` parameter in `AircraftPerformance` models if provided in the `source.attrs` data.
- No longer use a reference mass `ref_mass` in `AircraftPerformance` models. This is replaced by the `takeoff_mass` parameter if provided, or calculated from operating empty operating mass, max payload mass, total fuel consumption mass, reserve fuel mass, and the load factor.

Fixes

- Remove the `fuel` parameter from the `Emissions` model. This is inferred directly from the `source` parameter in `Emissions.eval`.
- Fix edge cases in the `jet.reserve_fuel_requirements` implementation. The previous version would return `nan` for some combinations of `fuel_flow` and `segment_phase` variables.
- Fix a spelling mistake: `units.kelvin_to_celcius` -> `units.kelvin_to_celsius`.

Internals

- Use `ruff` in place of `pydocstyle` for linting docstrings.
- Use `ruff` in place of `isort` for sorting imports.
- Update the `AircraftPerformance` template based on the patterns used in the new `PSFlight` class. This may change again in the future.

Support experimental interpolation against gridded specific humidity. Add new data-driven humidity scaling models.

Features

- Add new experimental `interpolation_q_method` field to the `ModelParams` data class. This parameter controls the interpolation methodology when interpolation against gridded specific humidity. The possible values are:
- `None`: Interpolate linearly against specific humidity. This is the default behavior and is the same as the previous behavior.
- `"cubic-spline"`: Apply cubic-spline scaling to the interpolation table vertical coordinate before interpolating linearly against specific humidity.
- `"log-q-log-p"`: Interpolate in the log-log domain against specific humidity and pressure.

This interpolation parameter is used when calling `pycontrails.core.models.interpolate_met`. It can also be used directly with the new lower-level `pycontrails.core.models.interpolate_gridded_specific_humidity` function.
- Add new experimental `HistogramMatching` humidity scaling model to match RHi values against IAGOS observations. The previous `HistogramMatchingWithEckel` scaling is still available when working with ERA5 ensemble members.
- Add new [tutorial](https://py.contrails.org/tutorials/interpolating-specific-humidity.html) discussing the new specific humidity interpolation methodology.

Breaking changes

- Add an optional `q_method` parameter to the `pycontrails.core.models.interpolate_met` function. The default value `None` agrees with the previous behavior.
- Change function signatures in the `cocip.py` module for consistency. The `interp_kwargs` parameter is now unpacked in the `calc_timestep_meterology` signature. Rename `kwargs` to `interp_kwargs` where appropriate.
- Remove the `cache_size` parameter in `MetDataset.from_zarr`. Previously this parameter allowed the user to wrap the Zarr store in a `LRUCacheStore` to improve performance. Changes to Zarr internals have broken this approach. Custom Zarr patterns should now be handled outside of `pycontrails`.

Fixes

- Recompute and extend quantiles for histogram matching humidity scaling. Quantiles are now available for each combination of `q_method` and the following ERA5 data products: reanalysis and ensemble members 0-9. This data is available as a parquet file and is packaged with `pycontrails`.
- Fix the precomputed Eckel coefficients. Previous values where computed for different interpolation assumptions and were not correct for the default interpolation method.
- Clip the scaled humidity values computed by the `humidity_scaling.eckel_scaling` function to ensure that they are non-negative. Previously, both relative and specific humidity values arising from Eckel scaling could be negative.
- Handle edge case of all NaN values in the `T_critical_sac` function in the `sac.py` module.
- Avoid extraneous copy when calling `VectorDataset.sum`.
- Officially support `numpy` v1.25.0.
- Set a `pytest-timeout` limit for tests in `tests/unit/test_ecmwf.py` to avoid hanging tests.
- Add `forecast_step` parameter to the `ACCF` model.

Internals

- Refactor auxillary functions used by `HistogramMatchingWithEckel` to better isolated histogram matching from Eckel scaling.
- Refactor `intersect_met` method in `pycontrails.core.models` to handle experimental `q_method` parameter.
- Include a `q_method` field in `Model.interp_kwargs`.
- Include precomputed humidity lapse rate values in the new `pycontrails.core.models._load_spline` function.
- Move the `humidity_scaling.py` module into its own subdirectory within `pycontrails/models`.