Nctoolkit

This is a minor release focusing on improving nctoolkit internals.

New methods and improvements

A new method, called ``check`` is introduced that can be used to troubleshoot data problems and to ensure there are no obvious data issues (such as a lack of CF-compliance).

Users can now access dataset calendars using ``ds.calendar``.

The ``drop`` method now lets you remove time steps using the ``times`` argument.

The dataset attribute `variables_detailed` is now removed after being replaced by `contents` in version 0.3.9.

Some coding improvements have enhanced the performance of the ``add``, ``subtract`` etc. methods.

Support for older versions of CDO

This version will recommend a CDO version of at least 1.9.7, because ensuring nctoolkit compatibility with earlier versions was becoming difficult and likely of little need to users.



Bug fixes

The methods ``multiply`` etc. failed when datasets did not have time as a dimension in version 0.4.0. This is now fixed. Previously, `ds.contents` always returned None for the number of time steps. Now fixed.

This is a major release that features some breaking changes.

Methods for adding, subtracting, multipling and dividing datasets from each other will be enhanced. Until now these methods used a simplistic approach where values from matching time steps were added to each other, etc. So if you are subtracting a 12 time step file from a dataset, only the first 12 time steps were subtracted from. However, often this is not what you want. For example, you might want to subtract data from a file which contains montly values for each year.

This version of nctoolkit updates these methods so that it can automatically figure out, in most cases, what kind of addition etc. it should carry out. For example, if you have a dataset which has monthly values for each year from 1950 to 1999, and use ``subtract`` to subtract the values from a file which contains annual means for each year from 1950, it will subtract the annual mean for 1950 from each month in 1950 and the the annual mean for 1951 from each month in 1951, and so on.

Users are now able to specify the numeric precision of datasets using ``ds.set_precision``. By default, nctoolkit uses the underlying netCDF files's data type. This is normally not a problem. However, when the data type is integer, this can cause problems. ``nc.open_data`` has been updated with this issue in mind. It will now warn users when the data type of the netCDF is integer, and it will suggest switching to float 'F64' or 'F32'.

The ``drop`` method has been enhanced. It now accepts day, month and year as arguments to enable dropping specific time periods. For example ``ds.drop(month = 2, day = 29)`` will remove leap days. Code written to use the old ``drop`` method will now fail, as keywords are now required.

The method ``surface`` has now been renamed ``top`` for consistency with ``bottom``. ``surface`` is deprecated and will be removed in a few months.

The ``split`` method now allows users to split datasets into multiple files by variable.

``ds.times`` now returns a datetime object, not a str as before.

This is a minor release focusing on under-the-hood improvements.

A new method, ``from_xarray`` is added for converting xarray datasets to nctoolkit datasets.

Methods for identifying how many missing values appear in datasets have been added: `na_count` and `na_frac`. These will identify the number or fraction of values that are missing values in each grid cell. The methods operate the same way as the temporal methods. So `ds.na_frac("year")` will give you the fraction of values are missing values each year.

Methods for better upscaling of datasets will be added: ``box_mean``, ``box_sum``, ``box_max``. This will allow you to upscale to, for example, each 10 by 10 grid box using the mean of that grid box. This is useful for upscaling things like population data where you want the upscaled grid boxes to represent the entirety of the grid box, not the centre.

Improvements to ``merge`` have been made. When variables are not included in all files nctoolkit will now only merge those in each file in a multi-file dataset. Previously it threw an error.

Functions for finding the times and months in netCDF files are now available: ``nc_years`` and ``nc_months`.

The attribute ``variables_detailed`` has been changed to ``contents``. It will also now give the number of time steps available for each variable.

``cdo_command`` now allows users to specify whether the CDO command used is an ensemble method. Previously methods applied on a file by file basis.

This is a minor release, focusing on under-the-hood improvements and introducing better handling of files with varying vertical layers.

New methods

A method, ``vertical_integration`` for calculating vertically integrated totals for netCDF data of the likes of oceanic data, where the vertical levels vary spatially, is introduced. ``vertical_mean`` has been improved and can now calculate vertical mean in cases where the cell thickness varies in space.

Deprecations

``merge_time`` is deprecated, and its functionality will be incorporated into ``merge``. So, following this release ensemble merging operations should use ``merge``.

Improvements

``open_url`` is now able to handle multiple urls. Previously it could only handle one.

Some under-the-hood improvements have been made to ``assign`` to ensure that truth statements do not occassionally throw an error.

**Full Changelog**: https://github.com/pmlmodelling/nctoolkit/compare/v0.3.7...v0.3.8

This is a minor release with new features and one bug fix.

New methods

New mathematical methods for simple operations on variables will be added: ``abs``, ``power``, ``square``, ``sqrt``, ``exp``, ``log`` and ``log10``. These methods match numpy names.

Bug fix

``assign`` previously did not accept ``log10`` in the lambda function. Now fixed.

This is a minor release.

New methods
New methods ``ensemble_mean`` and ``ensemble_stdev`` are introduced for calculating variance and standard deviation across ensembles.

Method renaming
The method ``tvariance`` is deprecated and is now renamed ``tvar`` for naming consistency.