Heat

Changes

Communication

- 1058 Fix edge-case contiguity mismatch for Allgatherv (by ClaudiaComito)

Contributors

ClaudiaComito, JuanPedroGHM

Changes

- 1048 Support PyTorch 1.13.0 on branch release/1.2.x (by github-actions)

🐛 Bug Fixes

- 1038 Lanczos decomposition `linalg.solver.lanczos`: Support double precision, complex data types (by ClaudiaComito)
- 1034 `ht.array`, closed loophole allowing `DNDarray` construction with incompatible shapes of local arrays (by Mystic-Slice)

Linear Algebra

- 1038 Lanczos decomposition `linalg.solver.lanczos`: Support double precision, complex data types (by ClaudiaComito)

🧪 Testing

- 1025 mirror repository on gitlab + ci (by mtar)
- 1014 fix: set cuda rng state on gpu tests for test_random.py (by JuanPedroGHM)

Highlights
- [906](https://github.com/helmholtz-analytics/heat/pull/906) PyTorch 1.11 support
- [595](https://github.com/helmholtz-analytics/heat/pull/595) Distributed 1-D convolution: `ht.convolve`
- [941](https://github.com/helmholtz-analytics/heat/pull/941) Parallel I/O: write to CSV file with `ht.save_csv`.
- [887](https://github.com/helmholtz-analytics/heat/pull/887) Binary operations between operands of equal shapes, equal `split` axes, but different distribution maps.
- Expanded memory-distributed [linear algebra](linalg), [manipulations](manipulations) modules.

Bug Fixes
- [826](https://github.com/helmholtz-analytics/heat/pull/826) Fixed `__setitem__` handling of distributed `DNDarray` values which have a different shape in the split dimension
- [846](https://github.com/helmholtz-analytics/heat/pull/846) Fixed an issue in `_reduce_op` when axis and keepdim were set.
- [846](https://github.com/helmholtz-analytics/heat/pull/846) Fixed an issue in `min`, `max` where DNDarrays with empty processes can't be computed.
- [868](https://github.com/helmholtz-analytics/heat/pull/868) Fixed an issue in `__binary_op` where data was falsely distributed if a DNDarray has single element.
- [916](https://github.com/helmholtz-analytics/heat/pull/916) Fixed an issue in `random.randint` where the size parameter does not accept ints.

New features

Arithmetics
- [945](https://github.com/helmholtz-analytics/heat/pull/945) `ht.divide` now supports `out` and `where` kwargs
Communication
- [868](https://github.com/helmholtz-analytics/heat/pull/868) New `MPICommunication` method `Split`
- [940](https://github.com/helmholtz-analytics/heat/pull/940) and [#967](https://github.com/helmholtz-analytics/heat/pull/967) Duplicate `MPI.COMM_WORLD` and `MPI_SELF` to make library more independent.

DNDarray
- [856](https://github.com/helmholtz-analytics/heat/pull/856) New `DNDarray` method `__torch_proxy__`
- [885](https://github.com/helmholtz-analytics/heat/pull/885) New `DNDarray` method `conj`
<a name="linalg"></a> Linear Algebra
- [840](https://github.com/helmholtz-analytics/heat/pull/840) New feature: `vecdot()`
- [842](https://github.com/helmholtz-analytics/heat/pull/842) New feature: `vdot`
- [846](https://github.com/helmholtz-analytics/heat/pull/846) New features `norm`, `vector_norm`, `matrix_norm`
- [850](https://github.com/helmholtz-analytics/heat/pull/850) New Feature `cross`
- [877](https://github.com/helmholtz-analytics/heat/pull/877) New feature `det`
- [875](https://github.com/helmholtz-analytics/heat/pull/875) New feature `inv`
Logical
- [862](https://github.com/helmholtz-analytics/heat/pull/862) New feature `signbit`
<a name="manipulations"></a> Manipulations
- [829](https://github.com/helmholtz-analytics/heat/pull/829) New feature: `roll`
- [853](https://github.com/helmholtz-analytics/heat/pull/853) New Feature: `swapaxes`
- [854](https://github.com/helmholtz-analytics/heat/pull/854) New Feature: `moveaxis`
Printing
- [816](https://github.com/helmholtz-analytics/heat/pull/816) New feature: Local printing (`ht.local_printing()`) and global printing options
- [816](https://github.com/helmholtz-analytics/heat/pull/816) New feature: print only on process 0 with `print0(...)` and `ht.print0(...)`
Random
- [858](https://github.com/helmholtz-analytics/heat/pull/858) New Feature: `standard_normal`, `normal`
Rounding
- [827](https://github.com/helmholtz-analytics/heat/pull/827) New feature: `sign`, `sgn`
Statistics
- [928](https://github.com/helmholtz-analytics/heat/pull/928) New feature: `bucketize`, `digitize`
General
- [876](https://github.com/helmholtz-analytics/heat/pull/876) Fix examples (Lasso and kNN)
- [894](https://github.com/helmholtz-analytics/heat/pull/894) Change inclusion of license file
- [948](https://github.com/helmholtz-analytics/heat/pull/948) Improve CSV write performance.
- [960](https://github.com/helmholtz-analytics/heat/pull/960) Bypass unnecessary communication by replacing `factories.array` with` DNDarray` contruct in random.py

- [864](https://github.com/helmholtz-analytics/heat/pull/864) Dependencies: constrain `torchvision` version range to match supported `pytorch` version range.

Highlights
- Slicing/indexing overhaul for a more NumPy-like user experience. Warning for distributed arrays: [breaking change!](breaking-changes) Indexing one element along the distribution axis now implies the indexed element is communicated to all processes.
- More flexibility in handling non-load-balanced distributed arrays.
- More distributed operations, incl. [meshgrid](https://github.com/helmholtz-analytics/heat/pull/794).

Breaking Changes
- [758](https://github.com/helmholtz-analytics/heat/pull/758) Indexing a distributed `DNDarray` along the `DNDarray.split` dimension now returns a non-distributed `DNDarray`, i.e. the indexed element is MPI-broadcasted.
Example on 2 processes:
python
a = ht.arange(5 * 5, split=0).reshape((5, 5))
print(a.larray)
>>> [0] tensor([[ 0, 1, 2, 3, 4],
>>> [0] [ 5, 6, 7, 8, 9],
>>> [0] [10, 11, 12, 13, 14]], dtype=torch.int32)
>>> [1] tensor([[15, 16, 17, 18, 19],
>>> [1] [20, 21, 22, 23, 24]], dtype=torch.int32)
b = a[:, 2]
print(b.larray)
>>> [0] tensor([ 2, 7, 12], dtype=torch.int32)
>>> [1] tensor([17, 22], dtype=torch.int32)
print(b.shape)
>>> [0] (5,)
>>> [1] (5,)
print(b.split)
>>> [0] 0
>>> [1] 0
c = a[4]
print(c.larray)
>>> [0] tensor([20, 21, 22, 23, 24], dtype=torch.int32)
>>> [1] tensor([20, 21, 22, 23, 24], dtype=torch.int32)
print(c.shape)
>>> [0] (5,)
>>> [1] (5,)
print(c.split)
>>> [0] None
>>> [1] None


Bug Fixes
- [758](https://github.com/helmholtz-analytics/heat/pull/758) Fix indexing inconsistencies in `DNDarray.__getitem__()`
- [768](https://github.com/helmholtz-analytics/heat/pull/768) Fixed an issue where `deg2rad` and `rad2deg`are not working with the 'out' parameter.
- [785](https://github.com/helmholtz-analytics/heat/pull/785) Removed `storage_offset` when finding the mpi buffer (`communication. MPICommunication.as_mpi_memory()`).
- [785](https://github.com/helmholtz-analytics/heat/pull/785) added allowance for 1 dimensional non-contiguous local tensors in `communication. MPICommunication.mpi_type_and_elements_of()`
- [787](https://github.com/helmholtz-analytics/heat/pull/787) Fixed an issue where Heat cannot be imported when some optional dependencies are not available.
- [790](https://github.com/helmholtz-analytics/heat/pull/790) catch incorrect device after `bcast` in `DNDarray.__getitem__`
- [796](https://github.com/helmholtz-analytics/heat/pull/796) `heat.reshape(a, shape, new_split)` now always returns a distributed `DNDarray` if `new_split is not None` (inlcuding when the original input `a` is not distributed)
- [811](https://github.com/helmholtz-analytics/heat/pull/811) Fixed memory leak in `DNDarray.larray`
- [820](https://github.com/helmholtz-analytics/heat/pull/820) `randn` values are pushed away from 0 by the minimum value the given dtype before being transformed into the Gaussian shape
- [821](https://github.com/helmholtz-analytics/heat/pull/821) Fixed `__getitem__` handling of distributed `DNDarray` key element
- [831](https://github.com/helmholtz-analytics/heat/pull/831) `__getitem__` handling of `array-like` 1-element key

Feature additions
Exponential
- [812](https://github.com/helmholtz-analytics/heat/pull/712) New feature: `logaddexp`, `logaddexp2`

Linear Algebra
- [718](https://github.com/helmholtz-analytics/heat/pull/718) New feature: `trace()`
- [768](https://github.com/helmholtz-analytics/heat/pull/768) New feature: unary positive and negative operations
- [820](https://github.com/helmholtz-analytics/heat/pull/820) `dot` can handle matrix-vector operation now

Manipulations
- [796](https://github.com/helmholtz-analytics/heat/pull/796) `DNDarray.reshape(shape)`: method now allows shape elements to be passed in as single arguments.

Trigonometrics / Arithmetic
- [806](https://github.com/helmholtz-analytics/heat/pull/809) New feature: `square`
- [809](https://github.com/helmholtz-analytics/heat/pull/809) New feature: `acosh`, `asinh`, `atanh`

Misc.
- [761](https://github.com/helmholtz-analytics/heat/pull/761) New feature: `result_type`
- [788](https://github.com/helmholtz-analytics/heat/pull/788) Added the partition interface `DNDarray` for use with DPPY
- [794](https://github.com/helmholtz-analytics/heat/pull/794) New feature: `meshgrid`
- [821](https://github.com/helmholtz-analytics/heat/pull/821) Enhancement: it is no longer necessary to load-balance an imbalanced `DNDarray` before gathering it onto all processes. In short: `ht.resplit(array, None)` now works on imbalanced arrays as well.

New features / Highlights
- [660](https://github.com/helmholtz-analytics/heat/pull/660) NN module for data parallel neural networks
- [699](https://github.com/helmholtz-analytics/heat/pull/699) Support for complex numbers; New functions: `angle`, `real`, `imag`, `conjugate`
- [702](https://github.com/helmholtz-analytics/heat/pull/702) Support channel stackoverflow
- [728](https://github.com/helmholtz-analytics/heat/pull/728) `DASO` optimizer
- [757](https://github.com/helmholtz-analytics/heat/pull/757) Major documentation overhaul, custom docstrings formatting

Bug fixes
- [706](https://github.com/helmholtz-analytics/heat/pull/706) Bug fix: prevent `__setitem__`, `__getitem__` from modifying key in place
- [709](https://github.com/helmholtz-analytics/heat/pull/709) Set the encoding for README.md in setup.py explicitly.
- [716](https://github.com/helmholtz-analytics/heat/pull/716) Bugfix: Finding clusters by spectral gap fails when multiple diffs identical
- [732](https://github.com/helmholtz-analytics/heat/pull/732) Corrected logic in `DNDarray.__getitem__` to produce the correct split axis
- [734](https://github.com/helmholtz-analytics/heat/pull/734) Fix division by zero error in `__local_op` with out != None on empty local arrays.
- [735](https://github.com/helmholtz-analytics/heat/pull/735) Set return type to bool in relational functions.
- [744](https://github.com/helmholtz-analytics/heat/pull/744) Fix split semantics for reduction operations
- [756](https://github.com/helmholtz-analytics/heat/pull/756) Keep track of sent items while balancing within `sort()`
- [764](https://github.com/helmholtz-analytics/heat/pull/764) Fixed an issue where `repr` was giving the wrong output.
- [767](https://github.com/helmholtz-analytics/heat/pull/767) Corrected `std` to not use numpy

DNDarray
- [680](https://github.com/helmholtz-analytics/heat/pull/680) New property: `larray`: extract local torch.Tensor
- [683](https://github.com/helmholtz-analytics/heat/pull/683) New properties: `nbytes`, `gnbytes`, `lnbytes`
- [687](https://github.com/helmholtz-analytics/heat/pull/687) New property: `balanced`

Factories
- [707](https://github.com/helmholtz-analytics/heat/pull/707) New feature: `asarray()`

I/O
- [559](https://github.com/helmholtz-analytics/heat/pull/559) Enhancement: `save_netcdf` allows naming dimensions, creating unlimited dimensions, using existing dimensions and variables, slicing

Linear Algebra
- [658](https://github.com/helmholtz-analytics/heat/pull/658) Bugfix: `matmul` on GPU will cast away from `int`s to `float`s for the operation and cast back upon its completion. This may result in numerical inaccuracies for very large `int64` DNDarrays

Logical
- [711](https://github.com/helmholtz-analytics/heat/pull/711) `isfinite()`, `isinf()`, `isnan()`
- [743](https://github.com/helmholtz-analytics/heat/pull/743) `isneginf()`, `isposinf()`

Manipulations
- [677](https://github.com/helmholtz-analytics/heat/pull/677) New features: `split`, `vsplit`, `dsplit`, `hsplit`
- [690](https://github.com/helmholtz-analytics/heat/pull/690) New feature: `ravel`
- [690](https://github.com/helmholtz-analytics/heat/pull/690) Enhancement: `reshape` accepts shape arguments with one unknown dimension
- [690](https://github.com/helmholtz-analytics/heat/pull/690) Enhancement: reshape accepts shape arguments with one unknown dimension.
- [706](https://github.com/helmholtz-analytics/heat/pull/706) Bug fix: prevent `__setitem__`, `__getitem__` from modifying key in place

Neural Networks
- [660](https://github.com/helmholtz-analytics/heat/pull/660) New submodule: `nn.DataParallel` for creating and training data parallel neural networks
- [660](https://github.com/helmholtz-analytics/heat/pull/660) New feature: Synchronous and Asynchronous gradient updates availble for `ht.nn.DataParallel`
- [660](https://github.com/helmholtz-analytics/heat/pull/660) New feature: `utils.data.datatools.DataLoader` for created a local `torch.utils.data.Dataloader` for use with `ht.nn.DataParallel`
- [660](https://github.com/helmholtz-analytics/heat/pull/660) New feature: `utils.data.datatools.Dataset` for created a local `torch.utils.data.Dataset` for use with `ht.nn.DataParallel`
- [660](https://github.com/helmholtz-analytics/heat/pull/660) Added MNIST example to `example/nn` to show the use of `ht.nn.DataParallel`. The `MNISTDataset` can be found in `ht.utils.data.mnist.py`
- [660](https://github.com/helmholtz-analytics/heat/pull/660) New feature: Data loader for H5 datasets which shuffles data in the background during training (`utils.data.partial_dataset.PartialH5Dataset`)
- [728](https://github.com/helmholtz-analytics/heat/pull/728) New feature: `nn.DataParallelMultiGPU` which uses `torch.distributed` for local communication (for use with `optim.DASO`)
- [728](https://github.com/helmholtz-analytics/heat/pull/728) New feature: `optim.DetectMetricPlateau` detects when a given metric plateaus.

Relational
- [792](https://github.com/helmholtz-analytics/heat/pull/792) API extension (aliases): `greater`,`greater_equal`, `less`, `less_equal`, `not_equal`

Statistical Functions
- [679](https://github.com/helmholtz-analytics/heat/pull/679) New feature: ``histc()`` and ``histogram()``

Types
- [712](https://github.com/helmholtz-analytics/heat/pull/712) New function: `issubdtype`
- [738](https://github.com/helmholtz-analytics/heat/pull/738) `iscomplex()`, `isreal()`


Bug fixes
- [709](https://github.com/helmholtz-analytics/heat/pull/709) Set the encoding for README.md in setup.py explicitly.
- [716](https://github.com/helmholtz-analytics/heat/pull/716) Bugfix: Finding clusters by spectral gap fails when multiple diffs identical
- [732](https://github.com/helmholtz-analytics/heat/pull/732) Corrected logic in `DNDarray.__getitem__` to produce the correct split axis
- [734](https://github.com/helmholtz-analytics/heat/pull/734) Fix division by zero error in `__local_op` with out != None on empty local arrays.
- [735](https://github.com/helmholtz-analytics/heat/pull/735) Set return type to bool in relational functions.
- [744](https://github.com/helmholtz-analytics/heat/pull/744) Fix split semantics for reduction operations
- [756](https://github.com/helmholtz-analytics/heat/pull/756) Keep track of sent items while balancing within `sort()`
- [764](https://github.com/helmholtz-analytics/heat/pull/764) Fixed an issue where `repr` was giving the wrong output.

Enhancements
Manipulations
- [690](https://github.com/helmholtz-analytics/heat/pull/690) Enhancement: reshape accepts shape arguments with one unknown dimension.
- [706](https://github.com/helmholtz-analytics/heat/pull/706) Bug fix: prevent `__setitem__`, `__getitem__` from modifying key in place
Unit testing / CI
- [717](https://github.com/helmholtz-analytics/heat/pull/717) Switch CPU CI over to Jenkins and pre-commit to GitHub action.
- [720](https://github.com/helmholtz-analytics/heat/pull/720) Ignore test files in codecov report and allow drops in code coverage.
- [725](https://github.com/helmholtz-analytics/heat/pull/725) Add tests for expected warnings.
- [736](https://github.com/helmholtz-analytics/heat/pull/736) Reference Jenkins CI tests and set development status to Beta.

- [678](https://github.com/helmholtz-analytics/heat/pull/678) Bugfix: Internal functions now use explicit device parameters for DNDarray and torch.Tensor initializations.
- [684](https://github.com/helmholtz-analytics/heat/pull/684) Bug fix: distributed `reshape` now works on booleans as well.