-----
Enhancements
~~~~~~~~~~~~
* **Advanced indexing**. The ``Array`` class has several new methods and
properties that enable a selection of items in an array to be retrieved or
updated. See the :ref:`tutorial_indexing` tutorial section for more
information. There is also a `notebook
<https://github.com/zarr-developers/zarr-python/blob/main/notebooks/advanced_indexing.ipynb>`_
with extended examples and performance benchmarks. :issue:`78`, :issue:`89`,
:issue:`112`, :issue:`172`.
* **New package for compressor and filter codecs**. The classes previously
defined in the :mod:`zarr.codecs` module have been factored out into a
separate package called `Numcodecs`_. The `Numcodecs`_ package also includes
several new codec classes not previously available in Zarr, including
compressor codecs for Zstd and LZ4. This change is backwards-compatible with
existing code, as all codec classes defined by Numcodecs are imported into the
:mod:`zarr.codecs` namespace. However, it is recommended to import codecs from
the new package, see the tutorial sections on :ref:`tutorial_compress` and
:ref:`tutorial_filters` for examples. With contributions by
:user:`John Kirkham <jakirkham>`; :issue:`74`, :issue:`102`, :issue:`120`,
:issue:`123`, :issue:`139`.
* **New storage class for DBM-style databases**. The
:class:`zarr.storage.DBMStore` class enables any DBM-style database such as gdbm,
ndbm or Berkeley DB, to be used as the backing store for an array or group. See the
tutorial section on :ref:`tutorial_storage` for some examples. :issue:`133`,
:issue:`186`.
* **New storage class for LMDB databases**. The :class:`zarr.storage.LMDBStore` class
enables an LMDB "Lightning" database to be used as the backing store for an array or
group. :issue:`192`.
* **New storage class using a nested directory structure for chunk files**. The
:class:`zarr.storage.NestedDirectoryStore` has been added, which is similar to
the existing :class:`zarr.storage.DirectoryStore` class but nests chunk files
for multidimensional arrays into sub-directories. :issue:`155`, :issue:`177`.
* **New tree() method for printing hierarchies**. The ``Group`` class has a new
:func:`zarr.hierarchy.Group.tree` method which enables a tree representation of
a group hierarchy to be printed. Also provides an interactive tree
representation when used within a Jupyter notebook. See the
:ref:`tutorial_diagnostics` tutorial section for examples. By
:user:`John Kirkham <jakirkham>`; :issue:`82`, :issue:`140`, :issue:`184`.
* **Visitor API**. The ``Group`` class now implements the h5py visitor API, see
docs for the :func:`zarr.hierarchy.Group.visit`,
:func:`zarr.hierarchy.Group.visititems` and
:func:`zarr.hierarchy.Group.visitvalues` methods. By
:user:`John Kirkham <jakirkham>`, :issue:`92`, :issue:`122`.
* **Viewing an array as a different dtype**. The ``Array`` class has a new
:func:`zarr.Array.astype` method, which is a convenience that enables an
array to be viewed as a different dtype. By :user:`John Kirkham <jakirkham>`,
:issue:`94`, :issue:`96`.
* **New open(), save(), load() convenience functions**. The function
:func:`zarr.convenience.open` provides a convenient way to open a persistent
array or group, using either a ``DirectoryStore`` or ``ZipStore`` as the backing
store. The functions :func:`zarr.convenience.save` and
:func:`zarr.convenience.load` are also available and provide a convenient way to
save an entire NumPy array to disk and load back into memory later. See the
tutorial section :ref:`tutorial_persist` for examples. :issue:`104`,
:issue:`105`, :issue:`141`, :issue:`181`.
* **IPython completions**. The ``Group`` class now implements ``__dir__()`` and
``_ipython_key_completions_()`` which enables tab-completion for group members
to be used in any IPython interactive environment. :issue:`170`.
* **New info property; changes to __repr__**. The ``Group`` and
``Array`` classes have a new ``info`` property which can be used to print
diagnostic information, including compression ratio where available. See the
tutorial section on :ref:`tutorial_diagnostics` for examples. The string
representation (``__repr__``) of these classes has been simplified to ensure
it is cheap and quick to compute in all circumstances. :issue:`83`,
:issue:`115`, :issue:`132`, :issue:`148`.
* **Chunk options**. When creating an array, ``chunks=False`` can be specified,
which will result in an array with a single chunk only. Alternatively,
``chunks=True`` will trigger an automatic chunk shape guess. See
:ref:`tutorial_chunks` for more on the ``chunks`` parameter. :issue:`106`,
:issue:`107`, :issue:`183`.
* **Zero-dimensional arrays** and are now supported; by
:user:`Prakhar Goel <newt0311>`, :issue:`154`, :issue:`161`.
* **Arrays with one or more zero-length dimensions** are now fully supported; by
:user:`Prakhar Goel <newt0311>`, :issue:`150`, :issue:`154`, :issue:`160`.
* **The .zattrs key is now optional** and will now only be created when the first
custom attribute is set; :issue:`121`, :issue:`200`.
* **New Group.move() method** supports moving a sub-group or array to a different
location within the same hierarchy. By :user:`John Kirkham <jakirkham>`,
:issue:`191`, :issue:`193`, :issue:`196`.
* **ZipStore is now thread-safe**; :issue:`194`, :issue:`192`.
* **New Array.hexdigest() method** computes an ``Array``'s hash with ``hashlib``.
By :user:`John Kirkham <jakirkham>`, :issue:`98`, :issue:`203`.
* **Improved support for object arrays**. In previous versions of Zarr,
creating an array with ``dtype=object`` was possible but could under certain
circumstances lead to unexpected errors and/or segmentation faults. To make it easier
to properly configure an object array, a new ``object_codec`` parameter has been
added to array creation functions. See the tutorial section on :ref:`tutorial_objects`
for more information and examples. Also, runtime checks have been added in both Zarr
and Numcodecs so that segmentation faults are no longer possible, even with a badly
configured array. This API change is backwards compatible and previous code that created
an object array and provided an object codec via the ``filters`` parameter will
continue to work, however a warning will be raised to encourage use of the
``object_codec`` parameter. :issue:`208`, :issue:`212`.
* **Added support for datetime64 and timedelta64 data types**;
:issue:`85`, :issue:`215`.
* **Array and group attributes are now cached by default** to improve performance with
slow stores, e.g., stores accessing data via the network; :issue:`220`, :issue:`218`,
:issue:`204`.
* **New LRUStoreCache class**. The class :class:`zarr.storage.LRUStoreCache` has been
added and provides a means to locally cache data in memory from a store that may be
slow, e.g., a store that retrieves data from a remote server via the network;
:issue:`223`.
* **New copy functions**. The new functions :func:`zarr.convenience.copy` and
:func:`zarr.convenience.copy_all` provide a way to copy groups and/or arrays
between HDF5 and Zarr, or between two Zarr groups. The
:func:`zarr.convenience.copy_store` provides a more efficient way to copy
data directly between two Zarr stores. :issue:`87`, :issue:`113`,
:issue:`137`, :issue:`217`.
Bug fixes
~~~~~~~~~
* Fixed bug where ``read_only`` keyword argument was ignored when creating an
array; :issue:`151`, :issue:`179`.
* Fixed bugs when using a ``ZipStore`` opened in 'w' mode; :issue:`158`,
:issue:`182`.
* Fill values can now be provided for fixed-length string arrays; :issue:`165`,
:issue:`176`.
* Fixed a bug where the number of chunks initialized could be counted
incorrectly; :issue:`97`, :issue:`174`.
* Fixed a bug related to the use of an ellipsis (...) in indexing statements;
:issue:`93`, :issue:`168`, :issue:`172`.
* Fixed a bug preventing use of other integer types for indexing; :issue:`143`,
:issue:`147`.
Documentation
~~~~~~~~~~~~~
* Some changes have been made to the Zarr Specification v2 document to clarify
ambiguities and add some missing information. These changes do not break compatibility
with any of the material as previously implemented, and so the changes have been made
in-place in the document without incrementing the document version number. See the
section on changes in the specification document for more information.
* A new :ref:`tutorial_indexing` section has been added to the tutorial.
* A new :ref:`tutorial_strings` section has been added to the tutorial
(:issue:`135`, :issue:`175`).
* The :ref:`tutorial_chunks` tutorial section has been reorganised and updated.
* The :ref:`tutorial_persist` and :ref:`tutorial_storage` tutorial sections have
been updated with new examples (:issue:`100`, :issue:`101`, :issue:`103`).
* A new tutorial section on :ref:`tutorial_pickle` has been added (:issue:`91`).
* A new tutorial section on :ref:`tutorial_datetime` has been added.
* A new tutorial section on :ref:`tutorial_diagnostics` has been added.
* The tutorial sections on :ref:`tutorial_sync` and :ref:`tutorial_tips_blosc` have been
updated to provide information about how to avoid program hangs when using the Blosc
compressor with multiple processes (:issue:`199`, :issue:`201`).
Maintenance
~~~~~~~~~~~
* A data fixture has been included in the test suite to ensure data format
compatibility is maintained; :issue:`83`, :issue:`146`.
* The test suite has been migrated from nosetests to pytest; :issue:`189`, :issue:`225`.
* Various continuous integration updates and improvements; :issue:`118`, :issue:`124`,
:issue:`125`, :issue:`126`, :issue:`109`, :issue:`114`, :issue:`171`.
* Bump numcodecs dependency to 0.5.3, completely remove nose dependency, :issue:`237`.
* Fix compatibility issues with NumPy 1.14 regarding fill values for structured arrays,
:issue:`222`, :issue:`238`, :issue:`239`.
Acknowledgments
~~~~~~~~~~~~~~~
Code was contributed to this release by :user:`Alistair Miles <alimanfoo>`, :user:`John
Kirkham <jakirkham>` and :user:`Prakhar Goel <newt0311>`.
Documentation was contributed to this release by :user:`Mamy Ratsimbazafy <mratsim>`
and :user:`Charles Noyes <CSNoyes>`.
Thank you to :user:`John Kirkham <jakirkham>`, :user:`Stephan Hoyer <shoyer>`,
:user:`Francesc Alted <FrancescAlted>`, and :user:`Matthew Rocklin <mrocklin>` for code
reviews and/or comments on pull requests.
.. _release_2.1.4: