Numpyro

New Features

- New distributions: [AsymmetricLaplace](https://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.continuous.AsymmetricLaplace) and [AsymmetricLaplaceQuantile](https://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.continuous.AsymmetricLaplaceQuantile) for quantile regression.
- New tutorials and examples:
+ [Bayesian neural network with SteinVI](https://num.pyro.ai/en/latest/examples/stein_bnn.html)
+ [Deep markov model inferred using SteinVI](https://num.pyro.ai/en/latest/examples/stein_dmm.html)
+ [Zero-Inflated Poisson regression model](https://num.pyro.ai/en/latest/examples/zero_inflated_poisson.html)

Enhancements and Bug Fixes

- 1320 Fix provenance logic for `numpyro.render_model`
- 1330 Fix vectorize sampling for sites with size=0
- 1331 Allow nested params in SteinVI
- 1333 Reduce memory consumption for SteinVI
- 1325 Fix compat MCMC signature

Thanks, Vinnie-Palazeti, wataruhashimoto52, hessammehr, OlaRonning, d-diaz!

New Features

- New VI inference: [SteinVI](https://num.pyro.ai/en/latest/contrib.html#stein-variational-inference). Checkout a couple examples in PRs 1297 1298 for the usage.
- New distributions: [MultivariateStudentT](https://num.pyro.ai/en/latest/distributions.html#multivariatestudentt), [DiscreteUniform](https://num.pyro.ai/en/latest/distributions.html#discreteuniform), [Kumaraswamy](https://num.pyro.ai/en/latest/distributions.html#kumaraswamy), [RelaxedBernoulli](https://num.pyro.ai/en/latest/distributions.html#relaxedbernoulli) .
- New tutorials and examples:
+ [Tutorial for Truncated distributions](https://num.pyro.ai/en/latest/tutorials/truncated_distributions.html): a complete guide for how to construct a NumPyro distribution.
+ [Bayesian Hierarchical Stacking case study](https://num.pyro.ai/en/latest/tutorials/bayesian_hierarchical_stacking.html) to average models based on weights from a hierarchical structure.
+ [Sine-skewed sine (bivariate von Mises) mixture](https://num.pyro.ai/en/latest/examples/ssbvm_mixture.html) to model the dihedral angles that occur in the backbone of a protein.
+ [AR2 processes](https://num.pyro.ai/en/latest/examples/ar2.html) to show how to avoid the (slow) Python for-loop.
+ [Holt-winter Exponential Smoothing](https://num.pyro.ai/en/latest/examples/holt_winters.html) example for time series forecasting.
+ [Hilbert space approximation for Gaussian processes](https://num.pyro.ai/en/latest/examples/hsgp.html) example is significantly revised.

Enhancements and Bug Fixes

- 1305 Fixes HMCECS bug for likelihoods with multiple plates
- 1304 Improves warning mechanism when plates are missing.
- 1301 Fixes sparse Poisson density sometimes returns int output.
- 1289 Make HMC Gibbs algorithms work with improper distributions
- 1284 Adds various KL divergences for Gamma/Beta families
- 1281 Raises error if there are duplicated deterministic sites
- 1271 Better warning mechanism with stacklevel
- 1270 Incorporate kl divergences of Tensorflow Probability distributions
- 1259 1266 Allow TruncatedNormal/Cauchy to take both low and high
- 1254 `numpyro.contrib.indexing` is moved to `numpyro.ops.indexing`
- 1252 Use multipledispatch for `kl_registry`
- 1250 Added `cdf` methods for gamma, inverse gamma, log normal densities
- 1248 Add ProvenanceArray to infer relational structure in a model
- 1244 Raise warning for the automatic enumeration behavior
- 1237 Enhance warnings for invalid parameters of `BetaProportion` distribution
- 1227 Allow `prior` to be callable in `random_flax_module` and `random_haiku_module`
- 1226 Allow init_to_sample work with scalar values
- 1225 Add color for divergences in Neal's example
- 1196 Allow custom precision function in laplace approximation autoguide
- 1194 Option to specify init state for SVI run
- 1185 1189 Avoid -inf/nan samples in truncated distributions
- 1182 Extend scope handler for plate stack frames
- 1179 Support enumerate support for zero inflated distributions
- 1169 Allow pickle autoguides

This release is composed of great contributions and feedback from the Pyro community: amalvaidya MarcoGorelli omarfsosa maw501 bjeffrey92 hessammehr OlaRonning dykim29 Carlosbogo wataruhashimoto52 Vedranh13 ahmadsalim austereantelope and many others. Thank you!

Breaking changes

Switch to softplus transforms for autoguide scales (thanks to [experiments](https://github.com/pyro-ppl/sandbox/tree/master/2021-03-softplus_scales) performed by vitkl).

New Features

- New autoguide: [AutoDAIS](http://num.pyro.ai/en/latest/autoguide.html#autodais) leverages HMC and annealed importance sampling within a variational inference framework
- New distributions: [MixtureSameFamily](http://num.pyro.ai/en/latest/distributions.html#mixturesamefamily), and directional distributions [SineBivariateVonMises](http://num.pyro.ai/en/latest/distributions.html#sinebivariatevonmises), [SineSkewed](http://num.pyro.ai/en/latest/distributions.html#sineskewed)
- New constraints: [l1_ball](http://num.pyro.ai/en/latest/distributions.html#l1-ball) for vectors with L1 norm less than 1
- New transforms: [L1BallTransform](http://num.pyro.ai/en/latest/distributions.html#l1balltransform), [SimplexToOrderedTransform](http://num.pyro.ai/en/latest/distributions.html#simplextoorderedtransform), [ScaledUnitLowerCholeskyTransform](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.transforms.ScaledUnitLowerCholeskyTransform)
- 1116 New [format_shapes](http://num.pyro.ai/en/latest/utilities.html#numpyro.util.format_shapes) utility to interpret the shapes of random variables/plates in a model.
- 1109 Allow direct use of TFP distributions in numpyro.sample
- New tutorials and examples:
+ [Principled prior with Dirichlet distribution](http://num.pyro.ai/en/latest/tutorials/ordinal_regression.html#Principled-prior-with-Dirichlet-Distribution) for Ordinal Regression case study
+ [Horseshoe regression](http://num.pyro.ai/en/latest/examples/horseshoe_regression.html)
+ [Bad posterior geometry and how to deal with it](http://num.pyro.ai/en/latest/tutorials/bad_posterior_geometry.html)

Enhancements and Bug Fixes

- 1108 Avoid numerical problems when using BernoulliProbs
- 1118 Recommend AutoNormal guide when hessian in AutoLaplace is singular
- 1126 Smarter warning about discrete inference in SVI models
- 1136 Support to use SA sampler with arviz
- 1139 Document Poisson `is_sparse` argument
- 1140 Make Sigmoid and StickBreakingTransform more stable
- 1149 Raise value error if num_steps bad in svi.run
- 1162 Use black[jupyter] in notebooks

This release is composed of great contributions and feedback from the Pyro community: MarcoGorelli OlaRonning d-diaz quattro svilupp peterroelants prashjet freddyaboulton tcbegley julianstastny alexlyttle and many others. Thank you!

This is a patch release with the following new feature and fixes:

+ New example [Hilbert space approximation of Gaussian processes](http://num.pyro.ai/en/stable/examples/hsgp.html) #1097 thanks to omarfsosa
+ Fix for rendering models with only discrete variables 1099 thanks to bdatko
+ Fix progress-bar issues when running multi-chain MCMC 1101

In 0.7.0 release, the wheel file uploaded to PyPI had some files not updated. This release fixes that issue.

Since this release, NumPyro can be installed along with the latest `jax` and `jaxlib` releases (their version restrictions have been relaxed). In addition, NumPyro will use the default JAX platform so if you installed JAX with GPU/TPU support, their devices will be used by default.

New Features

- New distributions: [SoftLaplace](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.continuous.SoftLaplace), [Weibull](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.continuous.Weibull), [BetaProportion](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.continuous.BetaProportion), [NegativeBinomial](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.conjugate.NegativeBinomial), [NegativeBinomial2](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.conjugate.NegativeBinomial2), [ZeroInflatedDistribution](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.discrete.ZeroInflatedDistribution), [ZeroInflatedPoisson](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.discrete.ZeroInflatedPoisson), [ZeroInflatedNegativeBinomial2](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.conjugate.ZeroInflatedNegativeBinomial2), [FoldedDistribution](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.distribution.FoldedDistribution)
- Support for DeepMind's [Optax](https://github.com/deepmind/optax) optimizers in [SVI](https://num.pyro.ai/en/latest/svi.html#numpyro.infer.svi.SVI)
- New ELBO objective: [TraceGraph_ELBO](http://num.pyro.ai/en/latest/svi.html#numpyro.infer.elbo.TraceGraph_ELBO) for non-reparameterized latent variables (e.g. discrete latent variables)
- A new wrapper [NestedSampler](https://num.pyro.ai/en/latest/contrib.html#numpyro.contrib.nested_sampling.NestedSampler) to leverage the nested sampling package [jaxns](https://github.com/Joshuaalbert/jaxns) for NumPyro models
- Implement `cdf` and `icdf` methods for many distributions
- New [cond](http://num.pyro.ai/en/latest/primitives.html#numpyro.contrib.control_flow.cond) primitive.
- New [infer_discrete](http://num.pyro.ai/en/latest/funsor.html#numpyro.contrib.funsor.discrete.infer_discrete) handler to sample discrete sites under enumeration. Check out the [annotation example](http://num.pyro.ai/en/latest/examples/annotation.html) for a usage.
- Structural mass matrix can be specified via [dense_mass](http://num.pyro.ai/en/latest/mcmc.html#numpyro.infer.hmc.NUTS) argument of the HMC/NUTS constructor 963
- New examples:
+ [Thompson sampling for Bayesian optimization with GPs](http://num.pyro.ai/en/latest/examples/thompson_sampling.html)
+ [Latent Dirichlet Allocation for topic modeling](http://num.pyro.ai/en/latest/examples/prodlda.html): a great example to illustrate the usage of Flax/Haiku in NumPyro

Enhancements and Bug Fixes

- Documentation and examples are greatly enhanced to make features more accessible
- Fix chain detection for various CPU device strings 1077
- Fix AutoNormal's `quantiles` method for models with non-scalar latent sites 1066
- Fix LocScaleReparam with `center=1` 1059
- Enhance auto guides to support models with deterministic sites 1022
- Support for mutable states in Flax and Haiku modules 1016
- Fix a bug in auto guides that happens when using the guide in Predictive 1013
- Support decorator syntax for effect handlers 1009
- Implement sparse Poisson log probability 1003
- Support `total_count=0` in Multinomial distribution 1000
- Add a flag to control regularize mass matrix behavior in mass matrix adaptation 998
- Add experimental Dockerfiles 996
- Allow setting max tree depth of NUTS sampler during warmup phase 984
- Fix dimensions mixed up in `ExpandedDistribution.sample` method 972
- MCMC objects can be pickled now 968

This release is made of great contributions and feedbacks from the Pyro community: ahoho, kpj, gustavehug, AndrewCSQ, jatentaki, tcbegley, dominikstrb, justinrporter, dirmeier, irustandi, MarcoGorelli, lumip, and many others. Thank you!