Chainer

This is the release note of v7.0.0b1. See [here](https://github.com/chainer/chainer/milestone/92?closed=1) for the complete list of solved issues and merged PRs.

Highlights

- *TabularDataset* is added. This is a new dataset interface that supports rich manipulation in a tabular form (like [pandas.DataFrame](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html)), e.g. loading only a specified subset of keys (columns), efficient slicing (with less transposition/concatenation), batch-wise preprocessing, etc. The API is still under development; we are adding more functionalities and widening its support in existing features where datasets are involved.

New Features
- Add interface to backprop from multiple variables (5952)
- Option to show progress bar during evaluation (6474, thanks wkentaro!)
- Elementwise `Power` for ChainerX (6496, thanks dido1998!)
- Add `chainerx.hstack`, `chainerx.vstack` and `chainerx.atleast_2d` (6886, thanks kshitij12345!)
- Add `TabularDataset` (7115)
- Add `TabularDataset.concat/join` (7116)
- Add `chainerx.expm1` and `chainerx.exp2` (7126, thanks aksub99!)
- Add `chainerx.log2` (7139)
- Add `TabularDataset.{transform/transform_batch}` (7150)
- Add `chainerx.log1p` (7161, thanks sky58!)
- Expose `chainerx::AsContiguous` as a public C++ API (7166)
- Emit warning on `chainerx` import in debug mode (7178)
- Add `chainer.as_array` for consistency with `chainer.as_variable` (7252, thanks tkerola!)
- Add `chainerx.moveaxis` (7265, thanks kshitij12345!)
- Add `chainerx.leaky_relu` (7351, thanks aksub99!)
- Add `chainerx.dstack` and `chainerx.atleast_3d` (7353, thanks kshitij12345!)
- Add Python operator `__abs__` with `chainerx.ndarray` (7364)
- Allow turning off the static subgraph optimizations using a config (7369)
- Add NumPy constants to ChainerX (7384)
- Add `chainerx.erf` (7404, thanks aksub99!)
- Add `align_corners` option to `resize_images` (7429)
- Add nearest mode to `resize_images` (7443)
- Add `input_device` to `StandardUpdater` (7472)
- Add `is_array_supported` method on `backend.Device` (7487)

Enhancements
- Refactor `roi_max_align_2d` and `roi_average_align_2d` (6405, thanks knorth55!)
- Support Tagged communication with `MPI_Status`. (6696, thanks y1r!)
- Support ChainerX in `F.copy` (6982)
- Avoid unnecessary updates in `F.batch_renormalization`, and related fixes (7104)
- Support ChainerX in `Variable.addgrad` (7132)
- Fix `cuda.DummyDevice` inheritance (7147)
- Add `Device.name` property (7149)
- `Link.serialize` to support ChainerX (7175)
- Fix typo in `Variable.backward` (7196)
- Call `require_grad()` on ChainerX `Variable.grad` setter (7198)
- Clear outputs in `FunctionNode.unchain` and raise error in ChainerX fallback mode (7216)
- Support ChainerX in `Variable.copydata` (7226)
- Support ChainerX in MNIST data parallel example (7227)
- `MultiprocessParallelUpdater` to support new devices (7245)
- Alias `StackVector<int64_t, kMaxNdim>` to `Dims` (7258)
- Support bool dtypes in `chainerx::{Max,Min}imum` (7261)
- Fix integral negative powers (7262)
- Make `chx.backward` not cause error even if backprop is not required (7287)
- Support `None` arguments in `chainerx.clip` and `chainerx.ndarray.clip` (7296)
- Support scalar in `chainerx::Where` (7325)
- `F.clip` function with `None` parameter to `min`/`max` (7333)
- Support cudnn deterministic max pooling (7390, thanks anaruse!)
- Avoid transferring from a native device to another in `Array::ToNative()` (7394)
- Add type hints to `Variable` (7400)
- Improve `get_device` error message when ChainerX is not available (7401)
- `get_device` to raise a more correct error types (7421)
- Make `EXEPECT_ARRAY_*` macros able to used outside ChainerX (7434)
- Add sequence support for ChainerX shape arguments (7446)
- Check positive dilation in `F.convolution_2d` (7448)
- Check positive dilation in `F.deconvolution_2d` (7449)
- Explicit check of chainerx arrays on fallback functions (7452)
- Support `F.copy` between non-ChainerX and ChainerX devices only if backprop is not required (7473)

Performance Improvements
- In `FunctionNode` ChainerX fallback, reuse `ChainerxDevice` taken from inputs to create outputs (7397)

Bug Fixes
- Fix type check of `F.where` (6872)
- Fix a bug in `Bernoulli.log_prob` (7064, thanks seiyab!)
- Fix uncopyable `MultiNodeBatchNormalization` (7106)
- Bugfix: `MultiNodeChainList` should not assume float32 (7165)
- Fix initialization of `L.Linear` when called with `n_batch_axes` (7167)
- Fix float16 and Tensor Core related issue in ChainerX (7189, thanks anaruse!)
- Fix recomputation of `L.BatchRenormalization` (7256)
- Fix `F.absolute_error` for ChainerX (7281, thanks crcrpar!)
- Fix a bug that root is ignored in scatter_dataset and bcast (7289)
- Fix condition to invoke cuDNN dropout (7293, thanks crcrpar!)
- Improve type check in `_values_to_dicts` so it works with unicode of python 2 too (7316)
- Fix DtypeError in `chainerx.square` (7321)
- Fix mypy errors (7423)
- Make `WeightDecay` aware of loss scale (7491)
- Fix `GradientMethod` ChainerX fallback for uninitialized parameters (7492)
- Bugfix for pytest 2x2 (7509)
- Fix AdamW update rule regression on CPU (7512)

Code Fixes
- Split binary functions from math.cc (7128)
- Avoid using `cuda.DummyDevice` and `cuda.get_device_from_array` (7148)
- Fix pointless comparison compiler warning in ChainerX (7160)
- Remove backslashes to continue lines of link targets (7170)
- Split trigonometric/hyperbolic routines from `math.cc` (7171)
- Remove duplicated code in `logic.cc` (7176)
- Consistent cases for Inplace (7181)
- Improve code in `testing.backend.BackendConfig` (7212)
- Split ChainerX statistics routines from `math.cc` (7222)
- Fix code style for long expressions (7231)
- Check device instance using `xp` when possible (7234)
- Move declaration of `AMax` and `AMin` to statistics routines (7269)
- Split reduction routines from `math.cc` (7270)
- Use `_` for private classes under `chainer.dataset.tabular` (7275)
- Remove unused using declaration (7284)
- Split misc routines from `math.cc` (7298)
- Fix wrong comment in ChainerX backward implementation (7311)
- Split explog routines from `math.cc` (7317)
- Fix style on imports (7338)
- Split rounding routines (7407)
- Split arithmetic ops from routines/math.h (7415)
- Put comments in `FindCuDNN.cmake` (7419)
- DRY optimizer test parameterizations (7437)
- Split logic routines from math (7444)
- Qualify some arguments of pool kernels `const&` (7453)
- Include `cuda_fp16.h` instead of `cuda_fp16.hpp` (7480)
- Use py::arg literal in ChainerX python binding (7490)
- Remove rounding kernels from math (7497)
- Rename and move activation routines from `math.h` (7501)
- Remove ChainerX `AsTypeKernel` (7522, thanks kshitij12345!)
- Split python binding math routines (7527)
- Use absolute namespace in macros (7536)

Documentation
- Improve contribution guide (6140)
- Fix dead sphinx links (6450)
- Fix `F.normalize` documentation (7062, thanks crcrpar!)
- Document `F.copy` view behavior (7135)
- Improve device documentation (7162)
- Document `backend.get_device_from_array` (7163)
- Remove `chainerx.md` (7179)
- Add `optimizers.MSVAG` to documentation (7183)
- Fix grammatical errors in documentation (7186)
- Fix capitalization of `F.relu` in doc (7188)
- Add missing doc entry for `CommunicatorBase.allgather` (7192)
- Fix invalid escape sequences in ChainerX routine docstrings (7214)
- Fix typos in `chainer.utils.type_check` (7249, thanks ktns!)
- Document `observe_value` and `observe_lr` trigger interval (7266)
- Fix `robots.txt` to allow indexing root (7306)
- Avoid installing ChainerX when building docs of other projects on ReadTheDocs (7363, thanks knorth55!)
- Improve `F.normalize` documentation (7371, thanks crcrpar!)
- Fix format of `static_graph.rst` (7389)
- Change Deformable Convolution 2D docs to match arguments (7402, thanks higumachan!)
- Avoid setting `test_iter.epoch` manually in the tutorial of training loop (7405)
- Remove "Comparison with other frameworks" from docs (7417)
- Fix documentation for `shape` in `generate_array` (7450)
- Remove test coverage from ChainerX contribution guide (7462)
- Correct Embed ID documentation (7484)
- Fix typo in `tabular_dataset.py` (7495, thanks nai62!)

Installation
- Fix ChainerX compilation with MSVC (7108, thanks durswd!)
- Allow `CUDNN_LIBNAME` to be specified by environment variable (7243)
- Use external `$MAKEFLAGS` instead if set in Travis CI script (7331)
- In `FindCuDNN.cmake`, prioritize explicit variables over environment variables (7441)
- Add ChainerX build option to use cuDNN from CuPy installation (7442)
- Pin `typing == 3.6.6` (7562)
- Fix `typing` requirements (7564)

Examples
- Add CIFAR example to ChainerMN (6839, thanks ai-kase!)
- Support device specifiers in MNIST data parallel example (6857)
- Support device specifiers in PTB example (7055)
- Support device specifiers in pix2pix example (7076)
- Support device specifiers in static graph example (7153)
- Support device specifiers in ImageNet data parallel example (7164)
- Support ChainerX in MNIST inference example (7169)
- Support device specifier in image captioning example (7204)
- Support device specifier in image captioning example (`predict.py`) (7206)
- Remove `PlotReport.available()` check in glance example (7209)
- Minor fix in DCGAN example README (7210)
- Fix sentiment example test (7215)
- Support device specifiers in MNIST model parallel example (7225)
- Use Agg backend in examples with plot functionality (7247)
- Support ChainerX in PTB gentxt example (7314)
- Support ChainerX in MNIST model parallel example (7330)
- Warn NaN in FP16 mode in dcgan example (7344)
- Warn NaN in FP16 mode in memnn example (7345)
- Warn NaN in FP16 mode in pix2pix example (7346)
- Warn NaN in FP16 mode in pos example (7354)
- Warn NaN in FP16 mode in reinforcement learning examples (7355)
- Warn NaN in FP16 mode in sentiment example (7356)
- Warn NaN in FP16 mode in static_graph_optimizations/cifar example (7357)
- Warn NaN in FP16 mode in static_graph_optimizations/mnist example (7358)
- Warn NaN in FP16 mode in vae example (7362)
- Warn NaN in FP16 mode in word2vec example (7366)
- Fix typo in wavenet example requirements (7367)
- Warn NaN in FP16 mode in wavenet example (7372)
- Support ChainerX in static subgraph optimization examples (7431)
- Implement `reset` method in the PTB example (7533)

Tests
- Add FP16 test to multi_node_chain_list (6575)
- [chainerx] Fix skipped_backward tests to return as PASS (6815, thanks kshitij12345!)
- Add configuration of new CI system (6843)
- Simplify `F.tensordot` test (6968, thanks ishanrai05!)
- Simplify `F.cumprod` test (6978, thanks hikjik!)
- Simplify `F.average` test (6995, thanks hikjik!)
- Move `test_cuda.py` to `backends_tests` (7144)
- Fix missing cuda in `chainerx.swapaxes` test (7184, thanks kshitij12345!)
- Split `Variable.grad` and `Variable.grad_var` tests (7191)
- Refactor `Variable.zerograd` test (7199)
- Add Tensor Core test for `chainerx.conv` and `chainerx.conv_transpose` (7203)
- Move `TestTanh` from `test_math.py` to test_trigonometric_hyperbolic.py (7207)
- Refactor `Variable.copydata` test (7224)
- Add a test to reproduce the bcast deadlock problem (7257)
- Add float16 comparison test (7260)
- Use `CUDA_VISIBLE_DEVICES` in ChainerX tests (7290)
- Add `chainer.as_array` test (7318)
- Rewrite `StandardUpdater` tests with pytest style assertion (7326)
- Change `0` to `0.0` for python2 (7373)
- Add missing parameter `dstack` to `invalid_shape` test (7457, thanks kshitij12345!)
- Use `pytest.mark.xfail` instead of `unittest.expectedFailure` (7488)

Others
- Remove "Research projects using Chainer" from README (7416)

This is the release note of v7.0.0a1. See [here](https://github.com/chainer/chainer/milestone/90?closed=1) for the complete list of solved issues and merged PRs.

Highlights

- Many examples including ImageNet, DCGAN and VAE start supporting ChainerX arrays

New Features

- Support orthogonal embedding initialization (6031)
- Add an option in `links.loss.CRF1d` to automatically sort the input sequence (6351)
- Add AdaBound (and AMSBound) (6388, thanks hitsgub!)
- Add `squared_difference` to chainerx (6501, thanks aksub99!)
- Implement array vs array functionality to `chainerx.minimum` (6541, thanks aksub99!)
- Add FP16 support to send/recv (6552)
- Implement array to array functionality to `chainerx.maximum` (6570, thanks aksub99!)
- Add Mean Var Python Bindings to ChainerX (6640, thanks kshitij12345!)
- Add `chainerx.ceil` (6705, thanks kshitij12345!)
- Add `chainerx.floor` (6707, thanks kshitij12345!)
- Add `chainerx.absolute` (6715, thanks dido1998!)
- Add `chainerx.argmin` and `chainerx.ndarray.argmin` (6740, thanks Harshan01!)
- Add `chainerx.amin` and `chainerx.min` (6752, thanks Harshan01!)
- Add `chainerx.a/sinh`,`chainerx.a/cosh` (6776, thanks kshitij12345!)
- Add `chainerx.fabs` and `chainerx.sign` (6777, thanks kshitij12345!)
- Add `chainerx.logical_and` `chainerx.logical_or` (6779, thanks kshitij12345!)
- Add `chainerx.all` and `chainerx.any` (6781, thanks kshitij12345!)
- Add `chainerx::Softmax` and `chainerx.softmax` (6814, thanks tohmae!)
- Add zero fill mode in allreduce of chainermn (6817)
- Make `BatchNorm` states public (6847)
- Introduce Native/CUDA macros for registering standard elementwise ops (6870, thanks kshitij12345!)
- Make adam variants more accessible (6874, thanks crcrpar!)
- Add `chainerx::Swapaxes` and `chainerx.swapaxes` (6897, thanks kshitij12345!)
- Add `chainerx.logical_xor` (7014, thanks ishanrai05!)
- Add `chainerx.log10` (7015, thanks ishanrai05!)
- Add `chainerx.isfinite` (7016, thanks kshitij12345!)
- Add bitwise ops to ChainerX (7017, thanks kshitij12345!)
- Add `chainerx.arctan2` (7028, thanks kshitij12345!)
- Add `chainerx.expand_dims` (7029, thanks kshitij12345!)
- Add `chainerx.flip`, `chainerx.fliplr` and `chainerx.flipud` (7065, thanks kshitij12345!)
- Add `chainerx.where` (7067, thanks kshitij12345!)
- Add `F.arctanh` (7095)

Enhancements

- Improve error message of `gradient_check.check_double_backward` (6427)
- Improve `link_hooks.SpectralNormalization` (6655, thanks crcrpar!)
- ChainerX Op registration: normalization (6719)
- ChainerX Op registration: arithmetic (6723)
- Implement Relu in ChainerX (6731, thanks dido1998!)
- Make device functions public (6744)
- ChainerX Op registration: creation (6745)
- ChainerX Op registration: linalg (6746)
- Allow `snapshot_object` have `condition` and `writer` option (6762)
- Support fallbacks of ChainerX on GetItem fail when indices contains `chainerx.ndarray` (6769)
- Fix `Evaluator` for `chainer.dataset.converter` (6768)
- Rename `patients` argument to `patience` in `EarlyStoppingTrigger` (6784)
- Remove `Backend` ctor and use `CreateBackend` (6785)
- ChainerX Op registration: pooling (6800)
- Define `__str__` for `Device` classes (6816, thanks nishnik!)
- Simplify `numeric.h` (6832)
- ChainerX Op registration: connection (6833)
- ChainerX Op registration: array members (6834)
- ChainerX Op registration: math (6842)
- Mixed dtypes: `chainerx::Minimum` (6858)
- Update `distributions.independent` (6860, thanks ganow!)
- Add `chainerx.ndarray.all` and `chainerx.ndarray.any` (6926)
- Fix `HuberLoss.forward` avoid loss of significance (6940)
- Support Tensor Core in `chainerx::Dot` (6960)
- Fix `F.get_item` backward for ChainerX (6991)
- Support NumPy scalars in ChainerX arithmetics (7004)
- Implement NumPy-like pairwise reduction for stability (7043, thanks grafi-tt!)
- Support mixed dtypes in `Stack` (7058)
- ChainerX Scalar / Array divisions (7075)
- Fix `Reshape` copy condition (7080)
- Fix trigger constructors to raise errors instead of assertion failures (7101)
- Support Tensor Core in `chainerx::Conv` (7112)

Performance Improvements

- Optimized ChainerX-to-CuPy `ndarray` conversion (6204)
- Use cuDNN in ReLU (6993)
- Fast integer scale unpooling (7114, thanks tkerola!)

Bug Fixes

- Avoid throwing in destructors (6725)
- Fix TypeError during BN deserialization on Win64 (6765, thanks hyabe!)
- Fix `chainerx.astype` casting from `float16` to `bool` in CUDA (6780, thanks kshitij12345!)
- Fix ArgMax of CUDA when all values are negative (6783)
- Fix unchain gradient pull (6804, thanks Rishav1!)
- Remove `chainerx.square` fallback since it is implemented in C++ (6823)
- Fix stack overflow caused when `to_gpu`/`to_cpu`/`to_intel64` were overridden (6824)
- Fix `filename` arg of `PlotReport` (6866)
- Make `InvalidType` picklable (6884, thanks zaltoprofen!)
- Rename the macro name for `AMinOp` (6922)
- Fix terminal column width retrieval in backprop traceback in Python 2 (6949)
- Avoid using ImportError during `import cupy` (6954)
- Fix cuDNN descriptor double destroy (6972)
- Fix `ConcatWithAsyncTransfer` (6992)
- Set `allow_pickle=True` (7036)
- Fix subview of zero-sized arrays (7037)
- Fix `At` output offset (7046)
- Fix handling of ndarray offsets (7047)
- Fix construction of `std::shared_ptr` with custom deleter in `chianer_interop.cc` (7107)
- Fix build with clang (7119)

Code Fixes

- Check headers with clang-tidy (6441)
- Refactor CUDA batch norm tensor descriptor (6724)
- Fix comments and add TODO to indexing routines (6789)
- Add `cuda_internal::DeviceInternals` to wrap handle etc. (6820)
- Clean up `DeviceInternals` (6827)
- Rename `CHAINERX_REGISTER_OP_{NATIVE,CUDA}` to `CHAINERX_{NATIVE,CUDA}_REGISTER_OP` (6865)
- Add comments on `del` (6933)
- Unify variable names in `gradient_check` (6935)
- Align macro parameter name (6941)
- Introduce `chainerx/kernels/` and rename existing device "op"s to "kernel"s (6944)
- Remove obsolete "Op" files (6959)
- Prefix macro with CHAINERX as per convention (7022)
- Use macro in exp_log.{cc/cu} (7068)
- Pass arguments by value in `native::Float16` and `cuda::Float16` (7069)
- Avoid importing object (7110)

Documentation

- Fix to clarify the description about initializer argument (6317)
- Add docs for two loss functions (6349, thanks hsezhiyan!)
- Improve docs of square, maximum and squared_difference (6451, thanks aksub99!)
- Append to v6 upgrade guide about Python 3.4 support drop (6493)
- Add reference and warning to `F.swish` document (6509, thanks fiarabbit!)
- Document fix in default initializer (6519)
- Convert utilities docs to one page (6595, thanks trancenoid!)
- Add `chainer.get_device` to doc (6735)
- Use search index (6881)
- Add `chainerx.sigmoid` docs (6889, thanks crcrpar!)
- Fix typo in `F.convolution_2d` (6890, thanks crcrpar!)
- Document `chainer.testing.LinkTestCase` (6895, thanks crcrpar!)
- Update README.txt for a link to the tutorial (6896)
- Fix broken link in `chainerx.md` (6899, thanks tkat0!)
- Document passive attributes in `FunctionTestCase` (6931)
- Fix documentation of renamed arguments (6932)
- Fix typo in `pickle_dataset.py` (6942)
- Update ChainerX contribution guide (6951)
- Support Sphinx 2.0 and use absolute path to support the latest RTD (7027)
- Fix link to ChainerMN docs in performance guide (7044)
- Update supported MPI list (7086)
- Document `CHAINERX_ENABLE_BLAS` environment variable (7098, thanks durswd!)
- Move backend docs to a separate page (7099)
- Document backend and device objects (7102)
- Remove extra spaces in docstrings (7125)
- Fix `AdamW` docstring (7137, thanks crcrpar!)
- Fix spelling of `AMSGrad` (7138, thanks crcrpar!)

Installation

- CMake for Windows(clang-cl) (7039, thanks durswd!)
- Exclude protobuf 3.8.0rc1 from dependencies (7083)

Examples

- Improve chainer examples (6399, thanks crcrpar!)
- Fix reinforcement_learning example to work with default dtype (6624)
- Support default dtype in vae example (6717)
- Support ChainerX in reinforcement learning example (6733)
- Support ChainerX in wavenet example (6736)
- Trivial fixes to Wavenet example (6737)
- Support ChainerX in VAE example (6739)
- Support ChainerX in text classification example (6769)
- Support ChainerX in DCGAN example (6773)
- Support ChainerX in word2vec example (6774)
- Show download progress bar in image-captioning example (6775)
- Support ChainerX in memnn example (6854)
- Use `filename` in PlotReport example (6880, thanks crcrpar!)
- Support ChainerX in CIFAR example (6936)
- Support ChainerX in POS-tagging example (7081)
- Support ChainerX in Sentiment example (7087)
- Add progress bar to sentiment analysis example (7103)
- Support ChainerX in Model Zoo example (7129)

Tests

- Simplify `F.mean_absolute_error` test (6253, thanks aksub99!)
- Simplify `F.bilinear` test (6488, thanks ishanrai05!)
- Simplify `F.deconvolution_2d` test (6498, thanks ishanrai05!)
- Display `pytest` summary (6625, thanks kshitij12345!)
- Travis test against v6 branch (6749)
- Fix Travis with macOS (6754)
- Dodge nondifferentiable inputs in `chainerx.max` test (6761)
- Make too slow initializers' tests faster (6792)
- Fix test failures in math test (6798)
- Simplify `F.flip` test (6801, thanks ishanrai05!)
- Simplify `F.where` test (6802, thanks ishanrai05!)
- Simplify `F.repeat` test (6803, thanks ishanrai05!)
- Fix `F.elu` test numeric error (6841)
- Relax tolerance for float16 in `unary_math_function_unittest` (6845)
- Relax tolerances and avoid non-differentiable points for FP16 in triplet loss tests (6855)
- Simplify `F.unpooling_nd` test (6861, thanks ishanrai05!)
- Simplify `F.local_response_normalization` test (6867, thanks ishanrai05!)
- Simplify `F.reshape` test (6868, thanks ishanrai05!)
- Simplify `F.layer_normalization` test (6871, thanks ishanrai05!)
- Fix test failure in `test_spatial_transformer_sampler.py` (6883)
- Simplify `F.prelu` test (6887, thanks ishanrai05!)
- Simplify `F.flatten` test (6888, thanks ishanrai05!)
- Simplify `F.dstack` test (6891, thanks ishanrai05!)
- Simplify `F.sign` test (6898, thanks hikjik!)
- Simplify `F.ceil` test (6900, thanks hikjik!)
- Simplify `F.floor` test (6901, thanks hikjik!)
- Fix `F.rrelu` test instability (6915)
- Fix `F.max_pooling_nd` test instability (6917)
- Fix flaky Huber loss test (6924)
- Simplify `F.fmod` test (6937, thanks hikjik!)
- Simplify `F.fix` test (6938, thanks hikjik!)
- Fix test parameters in ChainerX math tests (6946)
- Increase the default columns in Travis CI (6948)
- Fold Travis test outputs (6961)
- Simplify 'F.min', 'F.max' test (6962, thanks hikjik!)
- Simplify 'F.exp', 'F.log' test (6963, thanks hikjik!)
- Simplify `F.expm1` test (6965, thanks hikjik!)
- Fix flaky ChainerX `max_pool` test (6975)
- Simplify `F.bias` test (6976, thanks hikjik!)
- Simplify `F.cumsum` test (6977, thanks hikjik!)
- Refactor `Variable.addgrad` test (6979)
- Simplify `F.cosh`, `F.sinh` test (6980, thanks hikjik!)
- Simplify `F.log1p` test (6981, thanks hikjik!)
- Simplify `F.linear_interpolate` test (6984, thanks hikjik!)
- Simplify `F.fft`, `F.ifft` test (6985, thanks hikjik!)
- Simplify `F.matmul` test (6987, thanks ishanrai05!)
- Fix flaky `TestLogSumExp` (6988)
- Fix flaky `TestMin` (6989)
- Simplify `F.get_item` test (6990)
- Simplify `F.inv`, `F.batch_inv` test (6994, thanks hikjik!)
- Simplify `F.batch_l2_norm_squared` test (6996, thanks hikjik!)
- Simplify `F.accuracy` test (7006, thanks hikjik!)
- Simplify `F.binary_accuracy` test (7007, thanks hikjik!)
- Simplify `F.r2_score` test (7008, thanks hikjik!)
- Simplify `F.permutate` test (7010, thanks hikjik!)
- Simplify `F.scatter_add` test (7012, thanks hikjik!)
- Simplify `F.separate` test (7013, thanks hikjik!)
- Simplify `F.logsumexp` test (7018, thanks hikjik!)
- Skip tests that fail with NumPy 1.16.3 (7021)
- Add broadcast test in `test_math.py` (7023)
- Fix flaky `chainerx.abs` test (7024)
- Remove ChainerX acceptance tests (7026)
- Fix flaky `chainerx.tan` test (7033)
- Display `pytest` summary (cont.) (7089)

Others

- Make it easier to copy the instruction in the issue template (6665)
- Make git ignore `chainerx/libchainerx.dylib` (6666)
- Add `.mergify.yml` (7074)
- Improve mergify configuration (7111)

This is the release note of v6.7.0. See [here](https://github.com/chainer/chainer/milestone/103?closed=1) for the complete list of solved issues and merged PRs.

As [announced previously](https://chainer.org/announcement/2019/08/21/python2.html), this is the final release of v6 series, which is the last version supporting Python 2.


Bug Fixes

- Add try/finally block to yield in `reporter.py` (8511)

Documentation

- Fix several documentation errors in `chainer.functions.rnn.*` (8530, thanks msakai!)

Tests

- Use `FunctionTestCase` to test `F.decov` (8523)
- Skip `chainerx.fromfile` test when dtype is `bool_` and mode is text (8524)

This is the release note of v6.6.0. See [here](https://github.com/chainer/chainer/milestone/101?closed=1) for the complete list of solved issues and merged PRs.

Bug Fixes
- Fix SCE with ChainerX and normalize (8311)
- Fix kernel of double backward of `max_pooling_2d` (8329)
- Fix ChainerX fallback condition in batch normalization (8368)
- Fix `optimizer_hooks.GradientHardClipping` for scalar array (8372)
- Fix pickling of optimizers (8417)
- Register uninitialized persistents (8446)

Enhancements
- Compute `F.negative_sampling` in fp32 for fp16 inputs (8309)
- Fix `optimizer_hooks.GradientHardClipping` for ChainerX (8377, thanks kshitij12345!)

Documentation
- Fix documentation of NStepGRUBase (8337, thanks msakai!)
- Fix n-step RNN docs (8402)
- Fix typo in `/examples/seq2seq/README.md` (8404, thanks tanaken0515!)
- Changes citation to new KDD paper (8418)
- Link to examples directory for the current branch (8423)
- Update expected messages of `type_check` errors (8456)
- Update requirements (8502)

Tests
- Fix Decorrelated Batch Normalization tests (8340)
- Add missing FlexCI configurations (8352)
- Use `LinkTestCase` for `L.GroupNormalization` (8355)
- Show pytest summary in flexCI (8369)
- Set `CHAINER_CI` in Travis CI (8373)
- Set `CHAINER_CI` in ChainerX tests in Jenkins (8375)
- Set `CHAINER_CI` in Chainer tests in FlexCI (8381)
- Print installed packages in pytest (8386)
- Print actual array values in `FunctionTest` modified input error (8388)
- Avoid non-differential point in `TestTriplet` (8396)
- Use different docker image for each base development branch (8401)
- Disable ChainerMN FlexCI tests on v6 (8411)
- Use `fix_random` in xfail backward tests (8457)
- Avoid ChainerX slow tests in Jenkins (8474)
- Use CuPy v6 in ChainerX test in Jenkins (8477)
- Skip some `Convolution2D` tests for older numpy versions (8478)
- Fix Travis Openssl Error in OSX (8480)
- Fix flaky test of `_modified_xlogx` (8486)
- Add error message for invalid base branch in pfnCI (8500)
- Adjust timeout and build memory usage in FlexCI (8503)

This is the release note of v6.5.0. See [here](https://github.com/chainer/chainer/milestone/99?closed=1) for the complete list of solved issues and merged PRs.


Enhancements

- Display ChainerX availability in `print_runtime_info` (7860)
- Fix CuPy installation detection error message (8278)



Bug Fixes

- Fix `__str__` of parameterized class (8184)



Code Fixes


- Update variable names for consistent naming convention (8307)



Documentation

- Add document print runtime info (8165)
- Fix RNN documentation (8203)
- Fix a typo in the Cauchy distribution page (8209, thanks nzw0301!)



Tests

- Increase CPU memory for test instance in PFN CI (7955)
- Fix overflow warning in div backward test (8188)
- Disable flaky test retry in flexCI (8191)
- Relax `BatchNormalization` backward test tolerances (8196)
- Fix comparison with casting in hdf5 serializer test (8198)
- Fix tests of `L.BatchRenormalization` and adjust tolerances (8200)
- Adjust `TestConvolution2DFunction::test_double_backward` fp16 tolerance (8201)
- Fix `attrs` version (8222)
- Fix caffe test with protobuf>=3.8 (8232)
- Relax tolerances in ChainerX unary math tests (8235)
- Add Jenkins extras (8252)
- Fix `HDF5Serializer` test for h5py<2.9 (8256)


Others

- Replace Slack invitation links (8284)

This is the release note of v6.4.0. See [here](https://github.com/chainer/chainer/milestone/98?closed=1) for the complete list of solved issues and merged PRs.

Enhancements

- Insert missing spaces between concatenated string literals (7935)
- Make parameterized test names deterministic (8134)


Bug Fixes

- Fix decorrelated batch normalization when groups ≠ 1 (7825)
- Support mixed16/float16 `GroupNormalization` (8113)
- Fix deadlock on `MultiprocessIterator` and `MultiprocessParallelUpdater` (8126)
- Fixes `deepcopy` for chain parameters (8150)

Code Fixes

- Remove unused argument from decorrelated batch norm (8097)

Documentation

- Add undocumented arguments to snapshot extension signature (8016)
- Add a note about incompatibility with NumPy 1.17 + Python2 (8028)
- Fix grammatical errors in documentation (8036)
- Fix doc of `backend.copyto` (8056)
- Fix typo (8161)

Installation

- Fix NumPy version in Dockerfile (8068)

Tests

- Refactor `DecorrelatedBatchNormalizationTest` and add stable input (7940)
- Relax float16 tolerances in `F.batch_inv` test (7981)
- Relax tolerances in old cuDNN convolution tests (7982)
- Fix numerical gradient precision in `F.squared_error` test (8012)
- Fix flaky `negative_sampling` (8019)
- Relax tolerances in `gradient_check` test (8021)
- Explicit `h5py.File` `mode` (8107)
- Fix eps in `Contrastive.backward` (8108)
- Remove duplicated parameter in `test_normal.py` (8117)
- Fix macOS Travis error caused by Homebrew (8118)
- Add timestamp to Travis CI log (8119)
- Relax tolerance of `im2col` test (8135)