Pyro-ppl

New features
- 2693 A [GumbelSoftmaxReparam](http://docs.pyro.ai/en/stable/infer.reparam.html#pyro.infer.reparam.softmax.GumbelSoftmaxReparam) for relaxed categorical distributions

Bug fixes
- 2683 Support PyTorch 1.7
- 2682 Fix `help(MyDistribution)`
- 2679 Fix `TraceEnum_ELBO.compute_marginals()`
- 2677 Warn if `infer_discrete()` finds no discrete sites

New features

- New Tutorials:
- [Deep Generative Modeling with Single Cell Data](http://pyro.ai/examples/scanvi.html)
- [Conditional Variational Autoencoder](http://pyro.ai/examples/cvae.html)
- [Forecasting with Dynamic Linear Models](http://pyro.ai/examples/forecasting_dlm.html)
- New distributions and transforms:
- [OrderedLogistic](http://docs.pyro.ai/en/latest/distributions.html#orderedlogistic) distribution and [OrderedTransform](http://docs.pyro.ai/en/latest/distributions.html#orderedtransform)
- [ConditionalMatrixExponential](http://docs.pyro.ai/en/latest/distributions.html#conditionalmatrixexponential) normalizing flow
- [ConditionalSplineAutoregressive](http://docs.pyro.ai/en/latest/distributions.html#conditionalsplineautoregressive) normalizing flow
- [contrib.forecast](http://docs.pyro.ai/en/latest/contrib.forecast.html#pyro.contrib.forecast.forecaster.Forecaster) now supports [HaarReparam](http://docs.pyro.ai/en/latest/infer.reparam.html#pyro.infer.reparam.haar.HaarReparam), [DiscreteCosineReparam](http://docs.pyro.ai/en/latest/infer.reparam.html#pyro.infer.reparam.discrete_cosine.DiscreteCosineReparam), and [poutine.trace](http://docs.pyro.ai/en/latest/poutine.html#pyro.poutine.handlers.trace) to record posterior samples of latent variables.
- [CompartmentalModel](http://docs.pyro.ai/en/latest/contrib.epidemiology.html#pyro.contrib.epidemiology.compartmental.CompartmentalModel) now supports a [.finalize()](http://docs.pyro.ai/en/latest/contrib.epidemiology.html#pyro.contrib.epidemiology.compartmental.CompartmentalModel.finalize) method to add likelihoods that couple states across time.
- Integration with [Funsor](https://github.com/pyro-ppl/funsor), an experimental intermediate language for probabilistic programming
- [pyro.contrib.funsor](http://docs.pyro.ai/en/latest/contrib.funsor.html) is a new backend for Pyro that aims to simplify the implementations of Pyro's most powerful inference engines. For details, see: [tutorial 1](http://pyro.ai/examples/contrib_funsor_intro_i.html), [tutorial 2](http://pyro.ai/examples/contrib_funsor_intro_ii.html), [example usage with `pyroapi`](https://github.com/pyro-ppl/pyro/blob/dev/examples/contrib/funsor/hmm.py)
- [poutine.collapse](http://docs.pyro.ai/en/latest/poutine.html#pyro.poutine.handlers.collapse) and [pyro.barrier](http://docs.pyro.ai/en/latest/primitives.html#pyro.primitives.barrier) provide experimental support for collapsing conjugate fragments within existing inference algorithms, using [Funsor](https://github.com/pyro-ppl/funsor) under the hood.

Breaking changes
- Require PyTorch 1.6.
- Drop support for Python 3.5; require Python 3.6+.
- Zero inflated distributions changed interface. 2643

Bug fixes & performance tweaks
- [pyro.factor](http://docs.pyro.ai/en/stable/primitives.html#pyro.primitives.factor) statements are now allowed in guides without warning. 2664
- Fix model-directed subsampling in autoguides. 2638
- Fix sample shape bug in [LKJCorrCholesky](http://docs.pyro.ai/en/stable/distributions.html#lkjcorrcholesky) distribution. 2617
- Speed up log-matmul-exp operations in discrete enumeration and [DiscreteHMM](http://docs.pyro.ai/en/stable/distributions.html#discretehmm). 2640
- Fix ``potential_fn`` issues in MCMC. 2591

New features

- A new [pyro.contrib.epidemiology](http://docs.pyro.ai/en/latest/contrib.epidemiology.html) module for discrete-state discrete-time stochastic compartmental models. #2426
- New tutorials on:
- [Normalizing flows](https://pyro.ai/examples/normalizing_flows_i.html)
- [Epidemiology](https://pyro.ai/examples/epi_intro.html)
- [PyroModule](https://pyro.ai/examples/modules.html)
- [MAP/MLE inference](https://pyro.ai/examples/mle_map.html)
- New transforms and normalizing flows:
- Monotonic rational quadratic [Spline](http://docs.pyro.ai/en/latest/distributions.html#spline) flow.
- A [SplineCoupling](http://docs.pyro.ai/en/latest/distributions.html#splinecoupling) flow.
- A [SplineAutogregressive](http://docs.pyro.ai/en/latest/distributions.html#splineautoregressive) flow.
- A [MatrixExponential](http://docs.pyro.ai/en/latest/distributions.html#matrixexponential) flow.
- Conditional versions of `AffineAutoregressive`, `Householder`, `NeuralAutogregressive`, `Spline`, and `GeneralizedChannelPermute` flows.
- A Haar wavelet [transform](http://docs.pyro.ai/en/latest/distributions.html#pyro.distributions.transforms.HaarTransform) and [reparameterizer](http://docs.pyro.ai/en/latest/infer.reparam.html#pyro.infer.reparam.haar.HaarReparam).
- `Permute` and `AffineCoupling` can operate on a specific dimension with `dim` keyword argument 2472
- New distributions:
- [CoalescentTimes](http://docs.pyro.ai/en/latest/distributions.html#coalescenttimes), [CoalescentTimesWithRate](http://docs.pyro.ai/en/latest/distributions.html#coalescenttimeswithrate), and [CoalescentRateLikelihood](http://docs.pyro.ai/en/latest/contrib.epidemiology.html#pyro.distributions.CoalescentRateLikelihood) for coalescent processes in phylogenetics.
- [TruncatedPolyGamma](http://docs.pyro.ai/en/latest/distributions.html#pyro.distributions.TruncatedPolyaGamma).
- [ExtendedBinomial](http://docs.pyro.ai/en/latest/distributions.html#extendedbinomial) and [ExtendedBetaBinomial](http://docs.pyro.ai/en/latest/distributions.html#extendedbetabinomial) with relaxed support.
- [ImproperUniform](http://docs.pyro.ai/en/latest/distributions.html#improperuniform) for expressing factor graphs.
- Improvements to MCMC:
- structured mass matrix adaptation 2425 2473
- arrowhead precision matrix adaptation 2465
- support for randomized init strategies like [init_to_generated](http://pyro.ai/examples/epi_intro.html) #2417
- Improvements to SMC:
- systematic resampling 2488
- resampling based on effective sample size (ESS) 2486
- A new [SplitReparam](http://docs.pyro.ai/en/latest/infer.reparam.html#pyro.infer.reparam.split.SplitReparam) allows different inference strategies to apply to different parts of a tensor random variable.
- A new initialization strategy [init_to_generated](http://docs.pyro.ai/en/latest/infer.autoguide.html#pyro.infer.autoguide.initialization.init_to_generated).
- A [RandomVariable](http://docs.pyro.ai/en/latest/contrib.randomvariable.html#pyro.contrib.randomvariable.random_variable.RandomVariable) container class to support method chaining syntax for transforming distributions 2448

Bug fixes

- Support sequential plates in RenyiELBO 2541
- Fixes to `AffineAutogregressive` 2504
- Fixes to `BatchNorm` `TransformModule` 2459
- Fixes to how some transforms handle parameters 2544
- Fixes to reraising logic that clean up error reporting during inference 2494
- [many other fixes](https://github.com/pyro-ppl/pyro/pulls?q=is%3Apr+is%3Aclosed+merged%3A%3E2020-04-06+merged%3A%3C%3D2020-07-18) to documentation and code

New features

- A new [Spline](http://docs.pyro.ai/en/latest/distributions.html#spline) transform which implements element-wise rational spline bijections of linear order.
- A new [ConditionalAffineCoupling](http://docs.pyro.ai/en/latest/distributions.html#conditionalaffinecoupling) transform which implements the affine coupling layer of RealNVP that conditions on an additional context variable.

Enhancements to the [pyro.contrib.forecast](http://docs.pyro.ai/en/stable/contrib.forecast.html) module

- Support drawing samples in batches.
- Add walltime to backtest to measure performance of model training and forecasting.
- Support more likelihood distributions: Geometric, NegativeBinomial, ZeroInflatedNegativeBinomial.

Bug fixes

- 2399 raises an error when HMC/NUTS is used for a model with subsampling.
- 2390 makes `PyroModule` compatible with `torch.nn.RNN`.
- 2388 allows unused params in CSIS inference.
- 2384 fixes some caching issues in calculation of `log_abs_det_jacobian` of TransformModules
- 2365 fixes a naming bug in `LocScaleReparam` whereby all loc-scale reparameterized sites shared a single centeredness parameter.
- 2355 makes `jit_compile=True` flag in HMC/NUTS work for models with `pyro.param` statements.

New features

- A new [AutoNormal](http://docs.pyro.ai/en/latest/infer.autoguide.html#pyro.infer.autoguide.AutoNormal) guide that supports data subsampling, thanks to patrickeganfoley.
- Data subsampling support for the [AutoDelta](http://docs.pyro.ai/en/latest/infer.autoguide.html#pyro.infer.autoguide.AutoDelta) guide.
- A new [pyro.subsample](http://docs.pyro.ai/en/latest/primitives.html#pyro.primitives.subsample) primitive to aide in subsampling.
- An [AutoNormalizingFlow](http://docs.pyro.ai/en/latest/infer.autoguide.html#pyro.infer.autoguide.AutoNormalizingFlow) autoguide.
- A new [pyro.contrib.forecast](http://docs.pyro.ai/en/latest/contrib.forecast.html) module for multivariate hierarchical heavy-tailed forecasting, together with three tutorials: [uivariate, heavy-tailed](http://pyro.ai/examples/forecasting_i.html), [state space models](http://pyro.ai/examples/forecasting_ii.html), and [hierarchical models](http://pyro.ai/examples/forecasting_iii.html).
- A tutorial on [Boosting Black Box Variational Inference](http://pyro.ai/examples/boosting_bbvi.html), thanks to lorenzkuhn, gideonite, sharrison5, and TNU-yaoy.
- A [NeuTraReparam](http://pyro.ai/examples/neutra.html) example.
- [PyroModule]()'s interface is now stable, no longer EXPERIMENTAL.
- Better validation for [GaussianHMM](http://docs.pyro.ai/en/latest/distributions.html#gaussianhmm) and [LinearHMM](http://docs.pyro.ai/en/latest/distributions.html#linearhmm) distributions.
- Miscellaneous new [GaussianHMM](http://docs.pyro.ai/en/latest/distributions.html#gaussianhmm) method to handle conjugacy.

Bug fixes

- 2345 remove pillow-simd dependency
- 2327 Make pyro.deterministic not warn when called outside of inference
- 2321 Support plates in RenyiELBO
- 2266 Fixes to transform handling in MCMC api

Patches [1.2.0](https://github.com/pyro-ppl/pyro/releases/tag/1.2.0) with the following bug fixes:
- Fix for MCMC with parallel chains using multiprocessing, where transforms to the latent sites' support was not being correctly stored.
- Other minor rendering related fixes for tutorials.