Average-minimum-distance

Added

- Convenience function `compare` added as a flexible, easy way to compare crystals by AMD or PDD. This addition also added the dependency `pandas` to the package.

- `PDD_pdist` and `PDD_cdist` now support parallel processing with the keyword argument `n_jobs`. The argument `verbose` has changed to accept an integer which is passed to `joblib.Parallel` to control the level of verbosity.

- Comparison by molecular centres added (``csd-python-api`` only). The readers now accept ``molecular_centres`` (default ``False``), which give the yielded `PeriodicSet` objects an attribute ``.molecular_centres``, an array of the centres of molecules. The `amd.compare` function also supports the `.molecular_centres` argument, if `True` the molecular centres will be used for comparison instead of atomic centres.

- AMD/PDD and EMD speed improvements.

- Support for reader `pycodcif` (CIF v2.0) in `CifReader` through `ase`, if installed.

Changed

- Significant changes were made to `amd.PeriodicSet`, so old versions of this object are no longer compatible with the package (versions after 1.3 are not backwards compatible). They no longer have a generic `.tags` dictionary for additional data.

- `progressbar2` is planned to be removed. It is still in use for PDD comparisons without parallel processing, otherwise `joblib.Parallel` provides its own progress bar.

Removed

- Module `pset_io` removed in favour of just using `pickle` on a list. `h5py` has consequently been removed as a dependency.

Added

- Functions `PDD()` and `finite_PDD()` now accept the optional argument `return_row_groups`, default `False`. If `True`, a tuple is returned `(pdd, groups)` where `groups[i]` contains the indices of the point(s) corresponding to `pdd[i]`. Note that these indices are for the asymmetric unit of the input set, whose indices in ``.motif`` are accessible through ``.asymmetric_unit``.

Changed

- Readers now read atomic numbers rather than symbols which are better for checking if types align with Earth mover's distance flows.

- Argument `folder` removed from `CifReader`, now whether the path is a directory is checked by `os.path.isdir`.

- Fixed issue where AMD/PDD calculations were producing a RuntimeWarning from NumPy.

Removed

- SDDs removed for now until theory is complete.

- include_if and extract_data removed from readers.

Added

- Reconstructing periodic sets from PDDs added. Recently it was shown the PDD is complete in a general position and it is possible to reconstruct a periodic set from its PDD with large enough k. The first (inefficient) implementation of this algorithm has be added in the `reconstruct` module.

- Higher-order PDDs (the order parameter of `amd.PDD()` and `amd.PDD_finite()`) are removed and replaced with `amd.SDD()` (simplex-wise distance distribution). This invariant is only appropriate for finite sets, but first-order SDDs are equivalent to PDDs. Comparing SDDs is possible with `amd.compare.SDD_EMD()`.

- `CifReader` can now read a folder of .cifs or other files with the optional folder argument (True/False).

Changed

- `amd.finite_AMD` and `amd.finite_PDD` changed to `amd.AMD_finite` and `amd.PDD_finite` for easier autocompletion.

- Function `amd.compare.neighbours_from_distance_matrix` moved to `amd.utils.neighbours_from_distance_matrix`.

- Changed how the readers handle disorder to be simpler. Now, if `disorder='skip'` then any disorder means the structure is skipped, even if the disordered atom(s) would have been removed e.g. by remove_hydrogens. If `disorder='all_sites'`, no warnings are printed for any overlapping sites in the structure.

Removed

- Higher-order PDDs (the order parameter of `amd.PDD()` and `amd.PDD_finite()`) are removed and replaced with `amd.SDD()` (simplex-wise distance distribution). This invariant is only appropriate for finite sets, but first-order SDDs are equivalent to PDDs.

- Removed several functions which were bloating the package. List of removed functions/modules: `auto`, `ccdc_utils`, `periodicset.PeriodicSet.to_dict`, `io.SetReader.extract_tags`, `compare.mst_from_distance_matrix`, `compare.filter`, `compare.AMD_mst`, `compare.PDD_mst`, `utils.extract_tags`, `utils.neighbours_df_dict`. Also removed parameter `k` in functions in `compare`, and the `verbose` parameter (instead controlled by `compare.set_verbose()`).

- `utils.ETA` is removed and replaced by the progressbar2 package which is now a dependency.

Added

- Higher-order PDDs added. The parameter `order` of `amd.PDD` selects the PDD order (changed from before when it controlled lexicographically sorting the rows, which has been changed to `lexsort`), it can be any int, default `order=1`, the regular PDD. This infinite sequence of invariants grow in complexity but contain more information.

- AMD and PDD functions for finite point sets, `amd.finite_AMD()` and `amd.finite_PDD()`. `amd.finite_PDD()` accepts the new `order` parameter.

- Documentation is now available on [readthedocs](https://average-minimum-distance.readthedocs.io/en/latest/).

Changed

- Parameter `order` of `amd.PDD` no longer refers to lexicographically ordering the rows, this has been changed to `lexsort`. Higher-order PDDs are now implemented and `order` refers to the order of the PDD (default 1).

- Fixed/reworked the `ccdc` Reader (especially for disorder). The disorder option 'ordered_sites' now does not remove all atoms with partial occupancy; rather it essentially (for the `ccdc` reader) uses the motif of entry.molecule instead of entry.disordered_molecule (removes atoms whose label ends with ?), so the result should look like some feasible ordered structure.

Added

- Optional parameter low_memory added to AMD comparison functions. Uses a slower but more memory efficient method, useful for larger collections of AMDs.

- The functions `amd.utils.extract_tags()` and `SetReader.extract_tags()` return a `dict` containing the scalar (strings and numbers) tags of the PeriodicSets in the list/reader. The format of the returned dictionary is easily passable to `pandas.DataFrame()`.

- To compliment the addition of `.to_dict()`, the `CifReader` and `CSDReader` now have ways to extract additional data from `ccdc.entry.Entry` objects (later `ase` CifBlocks). The parameter `extract_data` can be passed as a dictionary with data (column) titles as keys and callable functions as values, which take the Entry object and return the data. E.g. if the custom function `get_density()` takes the entry and returns the density, use `extract_data={'Density': get_density}`, then the PeriodicSets will have the tag 'Density' with a value given by get_density(entry). This approach is flexible to any function, but some common ones are included in `amd.io.ccdc_utils`.

- `CifReader` and `CSDReader` accept `include_if` parameter, which can be `None` or a list of callable objects. This can be used to discard structures with specific properties. If `include_if` is a list, the `Entry`/`CifBlock` is passed to each callable and is skipped if any do not return `True`.

Changed

- The main README has been simplified; the old one read more like documentation than an introduction to the package. Details in the old README have moved to dedicated documentation.

- Bug fixes: infinite recursion in `PeriodicSet.__getattr__`; undetected equivalent sites with `ase` reader.

Removed

- Deprecated functions `amd.compare()` and the lower case versions of functions e.g. `amd.amd()`, `amd.amd_pdist()`, etc, have been removed. Also removed is the deprecated optional parameter `types=True` for the readers; atomic types are now always read.

- The option to pass a range of k values to comparison functions, along with the optional parameter `ord`, have been removed. Comparing over a range of k can be useful, but often its not if the range of distances aren't stored. No computational savings were really made in the function as opposed to a naive approach.

Added

- SetReader has additional convenient functions `.keys()` which generates all names (keeping order) and `.family(refcode)` which generates only items whose name starts with the string `refcode`.

- `CSDReader` has the function `.entry(refcode)` which returns a `PeriodicSet` given a `ccdc` refcode. This can be any refcode, not just those given when the reader is initialised.

Changed

- EMD has updated internals which are 2-3x faster.