Cupy

This is the release note of v4.1.0. See [here](https://github.com/cupy/cupy/milestone/26?closed=1) for the complete list of solved issues and merged PRs.

Enhancements

- Add NumPy-compatibility constants (1205, thanks keisuke-umezawa!)
- Support complex constants and functions in fuse (1207)
- Free pooled memory when `cufftMakePlan1d` cannot allocate memory (1236)
- Rename `CuFftError` to `CuFFTError` (1244)

Bug Fixes

- Fix regex in `einsum` to match empty input subscript (1186)
- Fix memory leak: mempool tried to find out-of-bounds bin when freeing chunk (1189)
- Fix scalar casting rule to support Windows (1194)
- Fix conversion from float16 to complex (1252)


Installation

- Separate NVTX module for better Windows support (1237)

Tests

- Separate tests for `cupy.power` against complex dtype (1187)
- Fix example test to pass on Windows (1188)
- Fix `real` and `imag` test for `bool` to pass on Windows (1192)
- Skip `int8.max` test on Windows due to NumPy bug (1193)
- Fix `hacking` version (1229)
- Fix 32-bit boundary test to support Windows (1255)

Others

- Fix `.gitignore` to exclude `.pyd` files (1227)

This is the major release of CuPy v4.0.0. All of the updates since the previous major version (v2.5.0) can be found in the release notes below:

This is the release candidate of v4. See [here](https://github.com/cupy/cupy/milestone/23?closed=1) for the complete list of solved issues and merged PRs.

Announcements
- We have started supporting CUDA9.1! A new wheel package `cupy-cuda91` is also available from this release. You can install it with `pip install cupy-cuda91`.
- The master branch has been switched to v5 development. The development of v4 will continue in the v4 branch.
- The major release of v4 is planned on Apr. 17.

New Features
- Implement cuDNN convolution interface (715)
- Support multi dimensional arrays in `solve` (845)
- Add `cupyx.rsqrt` (846)
- Add `destroy` method to `NcclCommunicator` (975)
- Implement `__setitem__` in fusion function (1002)

Bug Fixes
- Fix overflow in indices when indexing (758, thanks yuyu2172!)
- Fix matrix multiplication when matrixes have duplicated entries (834)
- Fix multithread bug with CUDA driver API (916)
- Remove trailing NULL from values returned from NVRTC (942)
- Fix `ndarray.diagonal` to accept appropriate argument of `axis2` (978, thanks ronekko!)
- Fix `eliminate_zeros` (998)
- Remove cudnn STATUS dict (1012)
- Fix temporary variables which are used when `input_num` is given (1020)
- Fix `cupy.copyto` ignore where argument when src is scalar (1028)

Installation
- Fix to use rpath only when wheel libs are specified (980)
- Update to CUDA 8.0 and use CuPy wheels in Dockerfiles (991)
- Support CUDA 9.1 (997)
- Fix exception handling fail on Windows with Python 3.x (1000)

Enhancements
- Improve matrix inverse speed using LU decomposition (695, 927, thanks stevendbrown!)
- Use CUDA version to decide if it import cuSOLVER or not (832)
- Simplify fp16 code in `carray.cuh` (870)
- Fix potential error at the stride for loop over the j-axis in `ReductionKernel` (874, thanks grafi-tt!)
- Hide Chunk class (933)
- Improve concatenate and other functions (949)
- Use `nogil` in FFT (950)
- Improve error message when import failed (970)
- Use current stream in array method (981)
- Change `group` argument name of `create_convolution_descriptor` (988)
- Use default stream in `_scatter_op` (989)
- Expose `CUDNN_BN_MIN_EPSILON` from `cudnn.h` (1011)

Documents
- Add upgrade guide for v2 & v4 (884)
- Add wheels to installation guide (955)
- Update array docstring (982, thanks juniorrojas!)
- Prefer `pip` in documentation (985)
- Add Docker update information to upgrade guide (993)
- Remove unnecessary heading from reference (996)
- Add URL to the directory in the documentation (999)
- Fix spelling mistake of NumPy and CuPy (1013)
- Fix typo (1015)
- Fix `scatter_add` docs (1025)
- Add complex dtypes on the overview (1026)
- Document more on CuPy/NumPy difference (1027)
- Add SciPy license to document (1037)
- Fix broken link to `numpy.sum` (1039)

Tests
- Free huge memory in slow test; Fix sum test to avoid contiguousness difference between CuPy and NumPy (971)
- Add AppVeyor configuration (1001)
- Fix shaped_random for complex number (1017)
- Skip cuDNN tests when cuDNN is unavailable (1041)
- Add Codecov.io configuration (1003)

Others
- Remove outdated TODO (1014)

This is the release of v4.0.0b4. See [here](https://github.com/cupy/cupy/milestone/20?closed=1) for the complete list of solved issues and merged PRs.

Important Changes
Starting from this release, CuPy supports wheel packages. The packages have different names depending on the CUDA versions. Installation is much faster with wheel packages than with the conventional installation from source.

To install CuPy from a wheel package, first uninstall the existing CuPy if you have, and then type the following command with appropriate CUDA version.

shell-session
$ pip install --pre cupy-cuda80
$ or
$ pip install --pre cupy-cuda90


Note: the wheel packages include cuDNN and NCCL2 binaries. These included binaries are automatically used by CuPy instead of those prepared by yourselves.

New Features
- Implement `isclose` (825)
- Introduce `cupyx` namespace (894)
- `cupy.scatter_add` is marked as deprecated. Use `cupyx.scatter_add` instead.
- Add logic type tests: `iscomplex`, `iscomplexobj`, `isfortran`, `isreal`, `isrealobj` (904)
- Import numpy complex dtypes into cupy namespace. (907, thanks ericmjl!)
- Simplify `free_all_blocks` interface (918)
- Support CUDA stream on FFT (926)
- Add forward/backward support for cuDNN clipped ReLU and ELU (938, thanks tkerola!)

Improvements
- Simplify `__getitem__` and `__setitem__` (724)
- Fix concatenate bug and improve performance (747)
- Improve `cupy.partition` performance (763)
- Support memory pool for FFT functions (910)
- Implement memory pool index compaction (912)
- Enable complex tensors for `cupy.matmul` and `cupy.einsum` (921, thanks hknerdgn!)
- Move `cupy.cufft` to internal functions of `cupy.fft.fft` (924)
- Use `cublas*gemmStridedBatched` APIs in matmul (930, thanks anaruse!)
- Use `__dealloc__` in memory object (934)
- Embed build-time version of CUDA/cuDNN (937)
- Remove unnecessary code in `cupy.fft` (951)
- Prefer data type objects over character codes (956)
- Prefer double quoted docstring to single quoted docstring (957)
- Simplify chained comparisons (958 )
- Remove warning in `cupy.core.fusion` (960)

Bug Fixes
- Fix `concatenate` bug and improve performance (747)
- Fix scalar broadcast (789)
- Test sparse with many types of sparse matrices (833)
- Fix array iteration index overflow (882)
- Fix get and set method with async mode (900)
- Fix bug about stream deletion (917)
- Fix test failure in cuDNN v5 which does not support `CUDNN_ACTIVATION_ELU` (959)

Installation
- Improve message during installation (762)
- Support wheel build (899)
- Exclude Cython files from sdist (920)
- Add `--cupy-long-description` option to inject long description for wheels (953)

Documentation
- Improve the document of `rand`/`randn` with examples (908)
- Fix typo: `nestead` -> `nested` (943, thanks rcalland!)
- Use https in supported websites (948)

Tests
- Test sparse with many types of sparse matrices (833)
- Fix test failure in cuDNN v5 which does not support `CUDNN_ACTIVATION_ELU` (959)

This is the release of v4.0.0b3. See [here](https://github.com/cupy/cupy/milestone/18?closed=1) for the complete list of solved issues and merged PRs.

Highlights
- `cupy.random.shuffle` performance has been improved. See 603 for details.
- More functions for sparse matrix have been added.

New Features
- Fast `shuffle` (603, thanks anaruse!)
- Implement `sum` for `cupy.sparse` (712)
- Implement conversion to complex scalar (783, thanks kohr-h!)
- Support scalar values in `atleast_nd` (819)
- Implement `size` function (827)
- Implement `eliminate_zeros` for `csc` matrix (831)
- Implement `rand` for `cupy.sparse` (836)
- Implement `coo` initializer for SciPy sparse matrix (857)

Improvements
- Accept `None` in `set_allocator` and `set_pinned_memory_allocator` (885)

Bug Fixes
- Fix `repeat` that behaved differently from NumPy (670)
- Fix return type of `linalg.norm` for complex input (781, thanks kohr-h!)
- Fix `ctxGetCurrent` (837)
- Make a copy when SVD is calculated (844)
- Fix memory pool to correctly detect being not established (856)
- Fix `out` argument in fusion (868)
- Fix `cupy.size` overflow in 32-bit (883)
- Fix downcast of size in `CArray` and `CIndexer` (892)

Examples
- Add an example of option pricing with multiple GPUs (513)
- Improve the conjugate gradient example (852)
- Fix typo (858, thanks juniorrojas!)

Documentation
- Add reference to paper (867)
- Fix docs to use NumPy 1.14 textual representation (876)
- Fix memory allocator docs (887)

Installation
- Change NVCC compiler options for CUDA9 (835, thanks anaruse!)

Tests
- Import `testing/parameterized.py` from Chainer (784)
- Remove unnecessary try catch (815)

This is the release of v4.0.0b2. See [here](https://github.com/cupy/cupy/milestone/15?closed=1) for the complete list of solved issues and merged PRs.

New Features
- LSQR (745, thanks KotaroSetoyama!)
- Incorporate persistent RNN functions of cuDNN v6 (549, thanks aonotas!)
- `cupy.moveaxis` (684, thanks fukatani!)
- `default_casting` option to ufuncs (720)

Improvements
- Support `None` as an argument for clip method (802)

Bug Fixes
- Fix `matmul` to raise `ValueError` on invalid shapes (737)
- Fix `RandomState.choice` reproducibility (741)
- Fix a bug in `stack` (749)
- Make `order=None` work like default order (764, thanks kohr-h!)
- Fix to correctly display filename of CUDA dump (777)
- Fix a bug about grouped convolution (785, thanks anaruse!)
- Fix `order=None` of `unravel_index` (791, thanks Hakuyume!)
- Cast `CUPY_SEED` environment variable to `uint64` (805, thanks toslunar!)

Tests
- Add test cases for tie of `argmin` and `argmax` (774)
- Fix checks for exceptions with inheritance (788)
- Skip a test using `numpy.stack` in numpy 1.9 (803)
- Skip a test using `numpy.matmul` in numpy 1.9 (817)

Documentation
- Improve dependency description in documentation (767)
- Fix typo in environment variable reference (769)
- Document Compute Capability requirement (772)
- Improve documentation of `ElementwiseKernel.__call__` (786)
- Add documentation for `accept_error` argument in `testing.numpy_cupy_raises` (787)
- Fix wrong documentation in `testing.numpy_cupy_equal` (804)
- Expose signatures of methods to reference (808)

Others
- Add pytest cache directory to gitignore (797)