Zappend

Latest version: v0.8.0

Safety actively analyzes 685525 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 1 of 2

0.8.0

* Added module `zappend.contrib` that contributes functions to
zappend's core functionality.

* Added experimental function `zappend.contrib.write_levels()` that generates
datasets using the
[multi-level dataset format](https://xcube.readthedocs.io/en/latest/mldatasets.html)
as specified by
[xcube](https://github.com/xcube-dev/xcube).
It resembles the `store.write_data(cube, "<name>.levels", ...)` method
provided by the xcube filesystem data stores ("file", "s3", "memory", etc.).
The zappend version may be used for potentially very large datasets in terms
of dimension sizes or for datasets with very large number of chunks.
It is considerably slower than the xcube version (which basically uses
`xarray.to_zarr()` for each resolution level), but should run robustly with
stable memory consumption.
The function requires `xcube` package to be installed. (19)

0.7.1

* The function `zappend.api.zappend()` now returns the number of slices
processed. (93)

* Moved all project configuration to `pyproject.toml` and removed
`setup.cfg` and requirements files. (88)

* Added new section _How do I ..._ to the documentation. (66)

* Fixed link to _slice sources_ in documentation main page.

* Fixed broken CI. (97)

0.7.0

* Made writing custom slice sources easier and more flexible: (82)

- Slice items can now be a `contextlib.AbstractContextManager`
so custom slice functions can now be used with
[contextlib.contextmanager](https://docs.python.org/3/library/contextlib.html#contextlib.contextmanager).

- Introduced `SliceSource.close()` so
[contextlib.closing()](https://docs.python.org/3/library/contextlib.html#contextlib.closing)
is applicable. Deprecated `SliceSource.dispose()`.

- Introduced new optional configuration setting `slice_source_kwargs` that
contains keyword-arguments passed to a configured `slice_source` together
with each slice item.

- Introduced optional configuration setting `extra` that holds additional
configuration not validated by default. Intended use is by a `slice_source`
that expects an argument named `ctx` and therefore can access the
configuration.

* Improved readability of the configuration reference by using setting
categories and applied logical ordering of settings within categories. (85)

0.6.0

Enhancements

* Added configuration setting `force_new`, which forces creation of a new
target dataset. An existing target dataset (and its lock) will be
permanently deleted before appending of slice datasets begins. (72)

* Chunk sizes can now be `null` for a given dimension. In this case the actual
chunk size used is the size of the array's shape in that dimension. (77)

API Changes

* Simplified writing of custom slice sources for users. The configuration setting
`slice_source` can now be a `SliceSource` class or any function that returns a
_slice item_: a local file path or URI, an `xarray.Dataset`,
a `SliceSource` object.
Dropped concept of _slice factories_ entirely, including functions
`to_slice_factory()` and `to_slice_factories()`. (78)

* Extracted `Config` class out of `Context` and made available via new
`Context.config: Config` property. The change concerns any usages of the
`ctx: Context` argument passed to user slice factories. (74)

0.5.1

* Fixed rollback for situations where writing to Zarr fails shortly after the
Zarr directory has been created. (69)

In this case the error message was
TypeError: Transaction._delete_dir() missing 1 required positional argument: 'target_path'.

0.5.0

Enhancements

* The configuration setting `attrs` can now be used to define dynamically
computed dataset attributes using the syntax `{{ expression }}`. (60)

Example:
yaml
permit_eval: true
attrs:
title: HROC Ocean Colour Monthly Composite
time_coverage_start: {{ lower_bound(ds.time) }}
time_coverage_end: {{ upper_bound(ds.time) }}


* Introduced new configuration setting `attrs_update_mode` that controls
how dataset attributes are updated. (59)

* Simplified logging to console. You can now set configuration setting
`logging` to a log level which will implicitly enable console logging with
given log level. (64)

* Added a section in the notebook `examples/zappend-demo.ipynb`
that demonstrates transaction rollbacks.

* Added CLI option `--traceback`. (57)

* Added a section in the notebook `examples/zappend-demo.ipynb`
that demonstrates transaction rollbacks.

Fixes

* Fixed issue where a NetCDF package was missing to run the
demo Notebook `examples/zappend-demo.ipynb` in
[Binder](https://mybinder.readthedocs.io/). (#47)

Page 1 of 2

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.