Numpyro

New Features

- 1394 New distribution Conditional Autoregressive [CAR](https://num.pyro.ai/en/latest/distributions.html?#car) (thanks to theorashid)
- 1434 New flexible auto guides for models with both global and local latent variables: [AutoSemiDAIS](https://num.pyro.ai/en/latest/autoguide.html#numpyro.infer.autoguide.AutoSemiDAIS) and [AutoSurrogateLikelihoodDAIS](https://num.pyro.ai/en/latest/autoguide.html#numpyro.infer.autoguide.AutoSurrogateLikelihoodDAIS)
- 1429 New example: [Conditional Variational Autoencoder in Flax](https://num.pyro.ai/en/latest/examples/cvae.html) (thanks to dirmeier)

Enhancements and Bug Fixes

- 1401 Fix `obs` argument is not respected when `sample` primitive is not executed under any handler (thanks to gcskoenig)
- 1412 TraceGraph_ELBO implementation using provenance tracking
- 1418 Fix SA sampler cannot be run in parallel chains
- 1419 Fix categorical sampler occasionally generate out-of-support samples
- 1436 Allow to use `potential_fn` in BarkerMH
- 1437 Fix for AutoMultivariateNormal.get_posterior method return incorrect distribution (thanks to xidulu)
- 1444 Promote shapes for observed variables inside `scan`'s transition function
- 1443 Consider the time dimension of markov models with `history=0` as plate
- 1441 More stable check for corr_cholesky constraint
- 1400 Fix SineBivariateVonMises sampler

New Features

- 1381 New `render_params` argument for [render_model](https://num.pyro.ai/en/latest/utilities.html#render-model)
- 1366 Allow a fixed number of steps in HMC.
- New tutorials and examples:
+ [Modelling mortality over space and time](https://num.pyro.ai/en/latest/examples/mortality.html)
+ New section on adding intermediate levels to [Bayesian Hierarchical Linear Regression](https://num.pyro.ai/en/latest/tutorials/bayesian_hierarchical_linear_regression.html#5.-Add-layer-to-model-hierarchy:-Smoking-Status) tutorial

Enhancements and Bug Fixes

- 1386 Support JAX 0.3.5
- 1388 Update jaxns to 1.0.0
- 1372 Fix batch shapes of `SineBivariateVonMises` distribution
- 1375 Add `__repr__` method for constraints
- 1358 Force validate_args to be keyword argument
- 1350 Enhance the AR2 example

Thanks, cstoafer, hesenp, tcbegley, themrzmaster, karm-patel, theorashid

New Features

- New distributions: [AsymmetricLaplace](https://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.continuous.AsymmetricLaplace) and [AsymmetricLaplaceQuantile](https://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.continuous.AsymmetricLaplaceQuantile) for quantile regression.
- New tutorials and examples:
+ [Bayesian neural network with SteinVI](https://num.pyro.ai/en/latest/examples/stein_bnn.html)
+ [Deep markov model inferred using SteinVI](https://num.pyro.ai/en/latest/examples/stein_dmm.html)
+ [Zero-Inflated Poisson regression model](https://num.pyro.ai/en/latest/examples/zero_inflated_poisson.html)

Enhancements and Bug Fixes

- 1320 Fix provenance logic for `numpyro.render_model`
- 1330 Fix vectorize sampling for sites with size=0
- 1331 Allow nested params in SteinVI
- 1333 Reduce memory consumption for SteinVI
- 1325 Fix compat MCMC signature

Thanks, Vinnie-Palazeti, wataruhashimoto52, hessammehr, OlaRonning, d-diaz!

New Features

- New VI inference: [SteinVI](https://num.pyro.ai/en/latest/contrib.html#stein-variational-inference). Checkout a couple examples in PRs 1297 1298 for the usage.
- New distributions: [MultivariateStudentT](https://num.pyro.ai/en/latest/distributions.html#multivariatestudentt), [DiscreteUniform](https://num.pyro.ai/en/latest/distributions.html#discreteuniform), [Kumaraswamy](https://num.pyro.ai/en/latest/distributions.html#kumaraswamy), [RelaxedBernoulli](https://num.pyro.ai/en/latest/distributions.html#relaxedbernoulli) .
- New tutorials and examples:
+ [Tutorial for Truncated distributions](https://num.pyro.ai/en/latest/tutorials/truncated_distributions.html): a complete guide for how to construct a NumPyro distribution.
+ [Bayesian Hierarchical Stacking case study](https://num.pyro.ai/en/latest/tutorials/bayesian_hierarchical_stacking.html) to average models based on weights from a hierarchical structure.
+ [Sine-skewed sine (bivariate von Mises) mixture](https://num.pyro.ai/en/latest/examples/ssbvm_mixture.html) to model the dihedral angles that occur in the backbone of a protein.
+ [AR2 processes](https://num.pyro.ai/en/latest/examples/ar2.html) to show how to avoid the (slow) Python for-loop.
+ [Holt-winter Exponential Smoothing](https://num.pyro.ai/en/latest/examples/holt_winters.html) example for time series forecasting.
+ [Hilbert space approximation for Gaussian processes](https://num.pyro.ai/en/latest/examples/hsgp.html) example is significantly revised.

Enhancements and Bug Fixes

- 1305 Fixes HMCECS bug for likelihoods with multiple plates
- 1304 Improves warning mechanism when plates are missing.
- 1301 Fixes sparse Poisson density sometimes returns int output.
- 1289 Make HMC Gibbs algorithms work with improper distributions
- 1284 Adds various KL divergences for Gamma/Beta families
- 1281 Raises error if there are duplicated deterministic sites
- 1271 Better warning mechanism with stacklevel
- 1270 Incorporate kl divergences of Tensorflow Probability distributions
- 1259 1266 Allow TruncatedNormal/Cauchy to take both low and high
- 1254 `numpyro.contrib.indexing` is moved to `numpyro.ops.indexing`
- 1252 Use multipledispatch for `kl_registry`
- 1250 Added `cdf` methods for gamma, inverse gamma, log normal densities
- 1248 Add ProvenanceArray to infer relational structure in a model
- 1244 Raise warning for the automatic enumeration behavior
- 1237 Enhance warnings for invalid parameters of `BetaProportion` distribution
- 1227 Allow `prior` to be callable in `random_flax_module` and `random_haiku_module`
- 1226 Allow init_to_sample work with scalar values
- 1225 Add color for divergences in Neal's example
- 1196 Allow custom precision function in laplace approximation autoguide
- 1194 Option to specify init state for SVI run
- 1185 1189 Avoid -inf/nan samples in truncated distributions
- 1182 Extend scope handler for plate stack frames
- 1179 Support enumerate support for zero inflated distributions
- 1169 Allow pickle autoguides

This release is composed of great contributions and feedback from the Pyro community: amalvaidya MarcoGorelli omarfsosa maw501 bjeffrey92 hessammehr OlaRonning dykim29 Carlosbogo wataruhashimoto52 Vedranh13 ahmadsalim austereantelope and many others. Thank you!

Breaking changes

Switch to softplus transforms for autoguide scales (thanks to [experiments](https://github.com/pyro-ppl/sandbox/tree/master/2021-03-softplus_scales) performed by vitkl).

New Features

- New autoguide: [AutoDAIS](http://num.pyro.ai/en/latest/autoguide.html#autodais) leverages HMC and annealed importance sampling within a variational inference framework
- New distributions: [MixtureSameFamily](http://num.pyro.ai/en/latest/distributions.html#mixturesamefamily), and directional distributions [SineBivariateVonMises](http://num.pyro.ai/en/latest/distributions.html#sinebivariatevonmises), [SineSkewed](http://num.pyro.ai/en/latest/distributions.html#sineskewed)
- New constraints: [l1_ball](http://num.pyro.ai/en/latest/distributions.html#l1-ball) for vectors with L1 norm less than 1
- New transforms: [L1BallTransform](http://num.pyro.ai/en/latest/distributions.html#l1balltransform), [SimplexToOrderedTransform](http://num.pyro.ai/en/latest/distributions.html#simplextoorderedtransform), [ScaledUnitLowerCholeskyTransform](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.transforms.ScaledUnitLowerCholeskyTransform)
- 1116 New [format_shapes](http://num.pyro.ai/en/latest/utilities.html#numpyro.util.format_shapes) utility to interpret the shapes of random variables/plates in a model.
- 1109 Allow direct use of TFP distributions in numpyro.sample
- New tutorials and examples:
+ [Principled prior with Dirichlet distribution](http://num.pyro.ai/en/latest/tutorials/ordinal_regression.html#Principled-prior-with-Dirichlet-Distribution) for Ordinal Regression case study
+ [Horseshoe regression](http://num.pyro.ai/en/latest/examples/horseshoe_regression.html)
+ [Bad posterior geometry and how to deal with it](http://num.pyro.ai/en/latest/tutorials/bad_posterior_geometry.html)

Enhancements and Bug Fixes

- 1108 Avoid numerical problems when using BernoulliProbs
- 1118 Recommend AutoNormal guide when hessian in AutoLaplace is singular
- 1126 Smarter warning about discrete inference in SVI models
- 1136 Support to use SA sampler with arviz
- 1139 Document Poisson `is_sparse` argument
- 1140 Make Sigmoid and StickBreakingTransform more stable
- 1149 Raise value error if num_steps bad in svi.run
- 1162 Use black[jupyter] in notebooks

This release is composed of great contributions and feedback from the Pyro community: MarcoGorelli OlaRonning d-diaz quattro svilupp peterroelants prashjet freddyaboulton tcbegley julianstastny alexlyttle and many others. Thank you!

This is a patch release with the following new feature and fixes:

+ New example [Hilbert space approximation of Gaussian processes](http://num.pyro.ai/en/stable/examples/hsgp.html) #1097 thanks to omarfsosa
+ Fix for rendering models with only discrete variables 1099 thanks to bdatko
+ Fix progress-bar issues when running multi-chain MCMC 1101