Mici

Minor release.

Bug fixes

* `[sampler].sample_chains_with_adaptive_warmup` methods added in previous release for sampling chains with adaptive warm up stage(s) previously were reinitialising each stage at the initial state rather than the final state from the previous stage. This is now fixed.

New features

* HTML representation added for `LabelledSequenceProgressBar` used for indicating progress through stages of adaptive chains to improve visualisation in interfaces supporting HTML output such as Jupyter notebooks.

New features and efficiency improvements

* Added framework for adaptation of transition parameters via classes in new `mici.adapters` module and associated `sample_chains_with_adaptive_warm_up` methods added to sampler classes. Initial adapters implemented are a dual averaging step size adapter, a diagonal metric adapter based on estimated variances and dense metric adapter based on estimated covariance matrices.
* New labelled sequence progress bar implementation to use with adaptive sampling chains and generalisation of progress bars to allow wrapping arbitrary sequences.
* Improved matrix operation efficiency. Repeated calls to matrix constructors avoided by caching matrices constructed by `transpose` / `sqrt` / `inv` methods and explicit matrix-vector multiplies used for inverted factored definite matrices to avoid bottleneck of `scipy.linalg.solve_triangular` which is much slower than a direct matrix-vector multiply.
* Remove redundant recaluclation of product of momentum and inverse metric when calculating value and gradient of `h2` Hamiltonian component.

Bug fixes

* Non-display of progress bars when running on Google Colab fixed.

Backwards incompatible changes

* Previously `ExplicitLeapfrogIntegrator` was aliased to `LeapfrogIntegrator`. `ExplicitLeapfrogIntegrator` has now been renamed to `LeapfrogIntegrator` with no aliases.
* The dictionaries of chain statistics returned by `sample_chain` and `sample_chains` call no longer contain an `hamiltonian` key. Instead by default the `traces` dictionary returned by these methods includes a `hamiltonian` entry (as generated by the default trace function used when a custom list of trace functions are not specified by the `trace_funcs` keyword argument). The rationale for this change is to standardize the chain statistics dictionary as only including values which don't correspond directly to a function of the chain states at the end of each iteration but instead generally depend on the whole transition, with the `trace_funcs` keyword argument already allowing recording of quantities outside this category.
* The dictionaries of chain statistics returned by `sample_chain` and `sample_chains` call no longer contain an `accept_prob` key due to the ambiguity of its meaning and differing usage in the static Metropolis and dynamic HMC integration transitions. Both classes of transitions now return a `accept_stat` statistic in the statistics dictionary, this corresponding to an overall measure of the probability of accepting a move both due to non-zero changes in the Hamiltonian over a trajectory **and** any early termination of trajectories due to convergence errors or non-reversible steps when using an implicit integrator. This single summary statistic is intended to both provide a quick summary of sampling performance e.g. in the monitored statistics in chain progress bars, and as the default target for adaptation of the integrator step size. Individual transitions also provide additional acceptance probability related statistics with differing meanings. The dynamic integration transitions (subclasses of `DynamicIntegrationTransition` and corresponding `Dynamic*HMC` sampler classes) additionally return an `av_metrop_accept_prob` statistic which is the average Metropolis acceptance probability for a move from the initial state to each of the states on the whole trajectory tree built (including subtrees which are invalid under the no-U-turn criteria); this does not include any adjustment for early termination of trajectories due to convergence errors / non-reversible steps in implicit integrators so should be used with caution in such cases. The Metropolis integration transitions (subclasses of `MetropolisIntegrationTransition` and corresponding `*MetropolisHMC` sampler classes) additionally return a `metrop_accept_prob` statistic which is the Metropolis acceptance probability of the move proposed by the transition without any adjustment for rejection due to convergence errors / non-reversible steps in implicit integrators.
* Documentation, internal naming and API has been standardized to use -ize rather than -ise endings. The only public facing change outside of the documentation is the renaming of `mici.systems.SoftAbsRegularisedPositiveDefiniteMatrix` to `mici.systems.SoftAbsRegularizedPositiveDefiniteMatrix`.

Minor release.

New features

* Implementation of dynamic integration time (NUTS-like) samplers has been refactored, with new implementation including both existing multinomial, and a new slice sampling, variants, with the latter corresponding to the original NUTS algorithm. An option to enable extra subtree termination checks to create more robust no-U-turn checking (see https://github.com/stan-dev/stan/pull/2800) has also been added, and enabled by default for the `DynamicMultinomialHMC` sampler.
* When used with Numpy v1.17 or above, Mici will now by default use the new-style random number generators for per-chain RNGs, with the new objects supporting more robust production of independent parallel random streams.
* Documentation of classes in `mici.systems` module is now much fuller.
* An option to disable display of progress bars when sampling chains was added.
* Display refreshes of progress bars rate limited to ~ 4 times per second to prevent the display updates becoming a bottleneck for very fast chains.
* Steffensen's method fixed point solver made more robust.
* Further matrix classes added to support low-rank update matrices and specialised matrix product classes, and matrix classes extended to be hashable to allow equality testing.
* Improved robustness to linear-algebra errors during sampling, with NumPy / SciPy `LinAlgError` exceptions when attempting Cholesky decomposition of non (numerically) positive definite matrix now handled rather than halting sampling.

Bug fixes

* Fixed bug in log determinant gradient implementation for triangular factored definite matrix classes.
* Fixed bug when integrating Gaussian constrained system with large timestep due to prior incorrect assumption of all eigenvalues of matrix being positive.

Minor release with several bug fixes. Thanks to kx-au for spotting and suggesting fixes to the bugs in the `ImplicitLeapfrogIntegrator` and `solve_fixed_point_direct` implementations.

* A previous attempt at adding a more helpful message when an error is encountered due to using parallel sampling with Autograd model functions without having `multiprocess` installed incorrectly assumed the exception raised would always be a `PicklingError` while in reality an `AttributeError` was being raised. The code now handles this case and correctly outputs an error message suggesting for the user to try installing `multiprocess` when trying to sample chains in parallel with Autograd model functions when `multiprocess` is not available.
* Several bugs / name errors in the implementation of the `mici.integrators.ImplicitLeapfrogIntegrator` have been fixed which previously prevented instantiation of this class.
* A bug in the naming of the tolerance parameter in `mici.solvers.solve_fixed_point_direct` introduced in 5e012034 has been fixed.
* The classes in the `mici.matrices` module have been refactored to use `abc` abstract base classes to make the expected interfaces for the various matrix types more explicit and docstrings added to most classes and methods.
* The `mici.matrices` unit tests have been updated to improve the coverage of the classes and reduce code duplication in writing the tests.
* The `mici.integrator` unit tests coverage has also been improved slightly by adding a basic test for a `ImplicitLeapfrogIntegrator` instance.

Minor release with improvements to non-linear equation solvers, additional block diagonal matrix classes and fix to bug when monitoring boolean chain statistics.

* Non-linear equation solvers, including solvers for projection on to manifold for constrained systems, all now implement a test for the change in the position vector between iterations being less than a tolerance as well as the tolerance on the non-linear equation being satisfied. For the projection solvers, controlling the position error in addition to the constraint error helps reduce the probability of spurious detections of non-reversible steps due to positions not converging to within the reverse check tolerance.
* Matrix classes corresponding to square block diagonal matrices added, allowing for example efficient use of metric matrix representations with such structure.
* Bug in update of running mean of monitored chain statistics due to assumption of floating point values fixed by casting all values to floats prior to computing update.

Minor release with sampling efficiency improvements, new facility for tracking the number of model function calls during sampling, improved robustness to use of non-NumPy arrays, and miscellaneous bug fixes.

* In constrained leapfrog integrator some derivatives of quantities are now pre-evaluated before the associated quantity is evaluated as often the quantity itself will be calculated alongside the derivative and cached. The resulting reduction in the number of model function evaluations provides a small gain in sampling efficiency in models where it applies.
* Chain state objects now optionally may be initialised with a `_count_calls` dictionary object which will be used to record the number of calls to system methods decorated with the `cache_in_state` and `multi_cache_in_state` decorators. This can be used to give a hardware independent measure of the operation cost of a sampling run.
* Code has been refactored to remove explicit dependence on the in-place semantics of the augmented assignment operators such as `+=` when acting on NumPy arrays. NumPy array like objects such as JAX `DeviceArray` instances do not perform updates in place, instead creating a new array object, meaning relying on such operations happening in place was silently failing when using such arrays.
* An non-guarded dependence on a decorator imported from `autograd` in the `autograd_wrapper` module has now been fixed meaning the package can be imported without `autograd` being installed as intended.
* Progress bar styling in Jupyter notebooks has been updated to remove the dependence on CSS variables defined in the `ipywidgets` stylesheets, instead now using explicit values.