Theano

=======================================

This release contains new features, improvements and bug fixes to prepare the upcoming release.

We recommend that every developer updates to this version.

Highlights:
- Make sure MKL uses GNU OpenMP

- **NB**: Matrix dot product (``gemm``) with ``mkl`` from conda
could return wrong results in some cases. We have reported the problem upstream
and we have a work around that raises an error with information about how to fix it.

- Optimized ``SUM(x^2)``, ``SUM(ABS(X))`` and ``MAX(ABS(X))`` operations with cuDNN reductions
- Added Python scripts to help test cuDNN convolutions
- Fixed invalid casts and index overflows in ``theano.tensor.signal.pool``

A total of 71 people contributed to this release since 0.9.0, see list below.

==========================================

This release contains new features, improvements and bug fixes to prepare the upcoming release candidate.

We recommend that every developer updates to this version.

Highlights:
- Announcing that `MILA will stop developing Theano <https://groups.google.com/d/msg/theano-users/7Poq8BZutbY/rNCIfvAEAwAJ>`_
- Bug fixes, crash fixes, warning improvements and documentation updates

A total of 70 people contributed to this release since 0.9.0, see list below.

Interface changes:
- Generalized ``AllocDiag`` for any non-scalar input

Convolution updates:
- Implemented fractional bilinear upsampling

cuDNN (GPU):
- Disallowed ``float16`` precision for convolution gradients
- Fixed memory alignment detection
- Added profiling in C debug mode (with theano flag ``cmodule.debug=True``)

New features:
- Implemented truncated normal distribution with box-muller transform
- Added ``L_op()`` overriding option for ``OpFromGraph``
- Added NumPy C-API based fallback implementation for ``[sd]gemv_`` and ``[sd]dot_``

Other more detailed changes:
- Improved stack trace follow-up for GPU optimizations
- Fixed gradient error for elemwise ``minimum`` and ``maximum`` when compared values are the same
- Fixed gradient for ``ARange``
- Removed ``ViewOp`` subclass during optimization

============================================

This release contains new features, improvements and bug fixes to prepare the upcoming release candidate.

We recommend that every developer updates to this version.

Highlights:
- conda packages now available and updated in our own conda channel ``mila-udem``.
To install it: ``conda install -c mila-udem -c mila-udem/label/pre theano pygpu``

- Improved elemwise operations

- Speed-up elemwise ops based on SciPy
- Fixed memory leak related to elemwise ops on GPU

- Fixed pygpu detection
- Bug fixes, crash fixes, warning improvements and documentation updates

A total of 69 people contributed to this release since 0.9.0, see list below.

Interface changes:
- Removed op ``ExtractDiag`` from ``theano.tensor.nlinalg``, now only in ``theano.tensor.basic``

Convolution updates:
- Added dilated causal convolutions for 2D

New features:
- Implemented ``topk`` and ``argtopk`` on CPU and GPU
- Added ``unravel_index`` and ``ravel_multi_index`` functions on CPU
- Implemented ``max()`` and ``min()`` functions for booleans and unsigned integers types

Others:
- Added ``R_op()`` for ``ZeroGrad``
- Added description for rnnblock

===========================================

This release contains new features, improvements and bug fixes to prepare the upcoming release candidate.

We recommend that every developer updates to this version.

Highlights:
- Support NumPy ``1.13``
- Support pygpu ``0.7``
- Added conda recipe
- Optional faster optimization step with new destroy handler
- Added documentation for RNNBlock
- Bug fixes, crash fixes, warning improvements and documentation updates

A total of 67 people contributed to this release since 0.9.0, see list below.

Interface changes:
- Added new parameter ``target`` for MRG functions

Convolution updates:
- Added unshared convolutions
- Added 3D separable convolutions
- Added 3D grouped convolutions
- Removed old ``conv3d`` interface
- Deprecated old ``conv2d`` interface
- Updated ``conv`` documentation

GPU:
- Added a meta-optimizer to select the fastest GPU implementations for convolutions

- cuDNN:

- Official support for ``v6.*`` and ``v7.*``, support for ``v5.*`` will be removed in next release
- Added spatial transformation operation based on cuDNN
- Updated and improved caching system for runtime-chosen cuDNN convolution algorithms
- Support cuDNN v7 tensor core operations for convolutions with runtime timed algorithms
- Restricted cuDNN reductions to contiguous inputs
- Automatic addition of cuDNN DLL path to ``PATH`` environment variable on Windows

New features:
- Added ``tensor6()`` and ``tensor7()`` in ``theano.tensor`` module
- Added boolean indexing for sub-tensors
- Added covariance matrix function ``theano.tensor.cov``
- Added new Theano flag ``pickle_test_value`` to help disable pickling test values

Others:
- Kept stack trace for optimizations in new GPU backend

Other more detailed changes:
- Moved all C code files into separate folder ``c_code`` in every Theano module
- Improvements for Jenkins tests

========================================

This release contains a lot of bug fixes, improvements and new features to prepare the upcoming release candidate.

We recommend that every developer updates to this version.

Highlights:
- Moved Python 3.* minimum supported version from 3.3 to 3.4
- Replaced deprecated package ``nose-parameterized`` with up-to-date package ``parameterized`` for Theano requirements
- Theano now internally uses ``sha256`` instead of ``md5`` to work on systems that forbide ``md5`` for security reason
- Removed old GPU backend ``theano.sandbox.cuda``. New backend ``theano.gpuarray`` is now the official GPU backend
- Support more debuggers for ``PdbBreakpoint``

- Scan improvements

- Speed up Theano scan compilation and gradient computation
- Added meaningful message when missing inputs to scan

- Speed up graph toposort algorithm
- Faster C compilation by massively using a new interface for op params
- Faster optimization step
- Documentation updated and more complete
- Many bug fixes, crash fixes and warning improvements

A total of 65 people contributed to this release since 0.9.0, see list below.

Interface changes:
- Merged duplicated diagonal functions into two ops: ``ExtractDiag`` (extract a diagonal to a vector),
and ``AllocDiag`` (set a vector as a diagonal of an empty array)
- Renamed ``MultinomialWOReplacementFromUniform`` to ``ChoiceFromUniform``

- Removed or deprecated Theano flags:

- ``cublas.lib``
- ``cuda.enabled``
- ``enable_initial_driver_test``
- ``gpuarray.sync``
- ``home``
- ``lib.cnmem``
- ``nvcc.*`` flags
- ``pycuda.init``

- Changed ``grad()`` method to ``L_op()`` in ops that need the outputs to compute gradient

Convolution updates:
- Extended Theano flag ``dnn.enabled`` with new option ``no_check`` to help speed up cuDNN importation
- Implemented separable convolutions
- Implemented grouped convolutions

GPU:
- Prevent GPU initialization when not required
- Added disk caching option for kernels
- Added method ``my_theano_function.sync_shared()`` to help synchronize GPU Theano functions
- Added useful stats for GPU in profile mode
- Added Cholesky op based on ``cusolver`` backend
- Added GPU ops based on `magma library <http://icl.cs.utk.edu/magma/software/>`_:
SVD, matrix inverse, QR, cholesky and eigh
- Added ``GpuCublasTriangularSolve``
- Added atomic addition and exchange for ``long long`` values in ``GpuAdvancedIncSubtensor1_dev20``
- Support log gamma function for all non-complex types
- Support GPU SoftMax in both OpenCL and CUDA
- Support offset parameter ``k`` for ``GpuEye``
- ``CrossentropyCategorical1Hot`` and its gradient are now lifted to GPU

- Better cuDNN support

- Official support for ``v5.*`` and ``v6.*``
- Better support and loading on Windows and Mac
- Support cuDNN v6 dilated convolutions
- Support cuDNN v6 reductions
- Added new Theano flags ``cuda.include_path``, ``dnn.base_path`` and ``dnn.bin_path``
to help configure Theano when CUDA and cuDNN can not be found automatically.

- Updated ``float16`` support

- Added documentation for GPU float16 ops
- Support ``float16`` for ``GpuGemmBatch``
- Started to use ``float32`` precision for computations that don't support ``float16`` on GPU

New features:
- Added a wrapper for `Baidu's CTC <https://github.com/baidu-research/warp-ctc>`_ cost and gradient functions
- Added scalar and elemwise CPU ops for modified Bessel function of order 0 and 1 from ``scipy.special``.
- Added Scaled Exponential Linear Unit (SELU) activation
- Added sigmoid_binary_crossentropy function
- Added tri-gamma function
- Added modes ``half`` and ``full`` for ``Images2Neibs`` ops
- Implemented gradient for ``AbstractBatchNormTrainGrad``
- Implemented gradient for matrix pseudoinverse op
- Added new prop `replace` for ``ChoiceFromUniform`` op
- Added new prop ``on_error`` for CPU ``Cholesky`` op
- Added new Theano flag ``deterministic`` to help control how Theano optimize certain ops that have deterministic versions.
Currently used for subtensor Ops only.
- Added new Theano flag ``cycle_detection`` to speed-up optimization step by reducing time spending in inplace optimizations
- Added new Theano flag ``check_stack_trace`` to help check the stack trace during optimization process
- Added new Theano flag ``cmodule.debug`` to allow a debug mode for Theano C code. Currently used for cuDNN convolutions only.

Others:
- Added deprecation warning for the softmax and logsoftmax vector case
- Added a warning to announce that C++ compiler will become mandatory in next Theano release ``0.11``

Other more detailed changes:
- Removed useless warning when profile is manually disabled
- Added tests for abstract conv
- Added options for `disconnected_outputs` to Rop
- Removed ``theano/compat/six.py``
- Removed ``COp.get_op_params()``
- Support of list of strings for ``Op.c_support_code()``, to help not duplicate support codes
- Macro names provided for array properties are now standardized in both CPU and GPU C codes
- Started to move C code files into separate folder ``c_code`` in every Theano module
- Many improvements for Travis CI tests (with better splitting for faster testing)
- Many improvements for Jenkins CI tests: daily testings on Mac and Windows in addition to Linux

==================================

This is a final release of Theano, version ``0.9.0``, with a lot of
new features, interface changes, improvements and bug fixes.

We recommend that everybody update to this version.