Gpytorch

Bug fixes

* MultitaskMultivariateNormal: fix tensor reshape issue by adamjstewart in https://github.com/cornellius-gp/gpytorch/pull/2081
* Fix handling of prior terms in ExactMarginalLogLikelihood by saitcakmak in https://github.com/cornellius-gp/gpytorch/pull/2039
* Fix bug in preconditioned KISS-GP / Hadamard Multitask GPs by gpleiss in https://github.com/cornellius-gp/gpytorch/pull/2090
* Add constant_constraint to ConstantMean by gpleiss in https://github.com/cornellius-gp/gpytorch/pull/2082

New Contributors
* mone27 made their first contribution in https://github.com/cornellius-gp/gpytorch/pull/2076

**Full Changelog**: https://github.com/cornellius-gp/gpytorch/compare/v1.8.0...v1.8.1

Major Features
* add variational nearest neighbor GP by LuhuanWu in https://github.com/cornellius-gp/gpytorch/pull/2026

New Contributors
* adamjstewart made their first contribution in https://github.com/cornellius-gp/gpytorch/pull/2061
* m-julian made their first contribution in https://github.com/cornellius-gp/gpytorch/pull/2054
* ngam made their first contribution in https://github.com/cornellius-gp/gpytorch/pull/2059
* LuhuanWu made their first contribution in https://github.com/cornellius-gp/gpytorch/pull/2026

**Full Changelog**: https://github.com/cornellius-gp/gpytorch/compare/v1.7.0...v1.8.0

**Important**: This release requires Python 3.7 (up from 3.6) and PyTorch 1.10 (up from 1.9)

New Features
- gpytorch.metrics module offers easy-to-use metrics for GP performance.(1870) This includes:
- gpytorch.metrics.mean_absolute_error
- gpytorch.metrics.mean_squared_error
- gpytorch.metrics.mean_standardized_log_loss
- gpytorch.metrics.negative_log_predictive_density
- gpytorch.metrics.quantile_coverage_error
- Large scale inference (using matrix-multiplication techniques) now implements the variance reduction scheme described in [Wenger et al., ICML 2022](https://arxiv.org/abs/2107.00243). (#1836)
- This makes it possible to use LBFGS, or other line search based optimization techniques, with large scale (exact) GP hyperparameter optimization.
- Variational GP models support online updates (i.e. “fantasizing new models). (1874)
- This utilizes the method described in [Maddox et al., NeurIPS 2021](https://papers.nips.cc/paper/2021/hash/325eaeac5bef34937cfdc1bd73034d17-Abstract.html)
- Improvements to gpytorch.priors
- New HalfCauchyPrior (1961)
- LKJPrior now supports sampling (1737)

Minor Features
- Add LeaveOneOutPseudoLikelihood for hyperparameter optimization (1989)
- The PeriodicKernel now supports ARD lengthscales/periods (1919)
- LazyTensors (A) can now be matrix multiplied with tensors (B) from the left hand side (i.e. B x A) (1932)
- Maximum Cholesky retries can be controlled through a setting (1861)
- Kernels, means, and likelihoods can be pickled (1876)
- Minimum variance for FixedNoiseGaussianLikelihood can be set with a context manager (2009)

Bug Fixes
- Fix backpropagation issues with KeOps kernels (1904)
- Fix broadcasting issues with lazily evaluated kernels (1971)
- Fix batching issues with PolynomialKernel (1977)
- Fix issues with PeriodicKernel.diag() (1919)
- Add more informative error message when train targets and the train prior distribution mismatch (1905)
- Fix issues with priors on ConstantMean (2042)

This release contains several bug fixes and performance improvements.

New Features
- Variational multitask models can output a single task per input (rather than all tasks per input) (1769)

Small fixes
- LazyTensorto method more closely matches the torch Tensor API (1746)
- Add type hints and exceptions to kernels to improve usability (1802)

Performance
- Improve the speed of fantasy models (1752)
- Improve the speed of solves and log determinants with KroneckerProductLazyTensor (1786)
- Prevent explicit kernel evaluation when expanding a LazyTensor kernel (1813)

Fixes
- Fix indexing bugs with kernels (1802, 1819, 1828)
- Fix cholesky bugs on CUDA (1848)
- Remove lines of code that generate warnings in PyTorch 1.9 (1835)

New features

- Add `gpytorch.kernels.PiecewisePolynomialKernel` (1738)
- Include ability to turn off diagonal correction for SGPR models (1717)
- Include ability to cast LazyTensor to half and float types (1726)


Performance improvements

- Specialty MVN log_prob method for Gaussians with sum-of-Kronecker covariances (1674)
- Ability to specify devices when concatenating rows of LazyTensors (1712)
- Improvements to LazyTensor symeig method (1725)


Bug fixes

- Fix to computing batch sizes of kernels (1685)
- Fix SGPR prediction when `fast_computations` flags are turned off (1709)
- Improve stability of `stable_qr` function (1714)
- Fix bugs with pyro integration for full Bayesian inference (1721)
- `num_classes` in `gpytorch.likelihoods.DirichletLikelihood` should be an integer (1728)

This release adds 2 new model classes, as well as a number of bug fixes:
- GPLVM models for unsupervised learning
- Polya-Gamma GPs for GP classification
In addition, this release contains numerous improvements to SGPR models (that have also been included in prior bug-fix releases).

New features
- Add example notebook that demos binary classification with Polya-Gamma augmentation (1523)
- New model class: Bayesian GPLVM with Stochastic Variational Inference (1605)
- Periodic kernel handles multi-dimensional inputs (1593)
- Add missing data gaussian likelihoods (1668)

Performance
- Speed up SGPR models (1517, 1528, 1670)

Fixes
- Fix erroneous loss for ExactGP multitask models (1647)
- Fix pyro sampling (1594)
- Fix initialize bug for additive kernels (1635)
- Fix matrix multiplication of rectangular ZeroLazyTensor (1295)
- Dirichlet GPs use true train targets not labels (1641)