Blksprs

Added

- Add ``disable_validation()`` function to disable validation

Changed

- Change output dtype of sparsity layouts from ``int32`` to ``bool``
- Apply ``.contiguous()`` to tensors for all ops

Added

- Add ``row_wise_max`` function
- Add ``row_wise_add`` function
- Add ``row_wise_sub`` wrapper function
- Expose functions in ``__init__.py``

Changed

- Refactor ``row_wise_sum`` function to ``misc`` module
- Change ``row_wise_sum`` to not make use of ``torch.autograd.Function``
- Rename ``broadcast_addition`` module to ``broadcast_ops``
- Rename ``broadcast_addition`` and ``broadcast_subtraction`` to ``broadcast_add`` and ``broadcast_sub``

Fixed

- Fix ``softmax`` shift trick not using per-row maximum values

Added

- Add ``repeat_interleave`` function

Fixed

- Fix ``undo_shape_blocksparse`` function checking the incorrect dimensions

Changed

- Downgrade Python version to 3.11

Added

- Add ``build_sparsity_layout_adaption`` function to create sparsity layout for adaption of sparsity block size
- Add ``adapt_layout`` function to adapt block-sparse tensor to new sparsity layout and sparsity block size

Changed

- ``to_sparse`` and ``to_dense`` will no longer return a copy of the input tensor if the input tensor is already in the
desired format

Added

- Add validation for sparsity of input for all applicable functions
- Add forward and backward functions for block-sparse ``gather`` operation
- Add forward and backward functions for block-sparse ``scatter_reduce`` operation (gradients only computable with
``reduce_op = 'sum'``)
- Add wrapper function ``scatter`` (applies ``scatter_reduce`` with ``reduce_op = 'none'``)
- Add ``build_distribution_layout`` function to create sparsity layout for distribution operations
- Add ``broadcast_addition`` and ``broadcast_subtraction`` functions

Changed

- Rename ``matmul_sss`` to ``matmul``
- Improve documentation

Fixed

- Fix memory leaks of backward passes