Opacus

New features
* Add a function of "double_backward" simplifying the training loop (661)

Bug fixes
* Fix issue with setting of param_group for the DPOptimizer wrapper (issue 649) (660)
* Fix issue of DDP optimizer for FGC. The step function incorrectly called "original_optimizer.original_optimizer" (662)
* Replace "opt_einsum.contract" by "torch.einsum"(663)

Bug fixes
* Make the import of opt_einsum.contract (linear.py) explicit (658)

New features
Fast Gradient Clipping and Ghost Clipping (656)

Bug fixes
* Fix gradient shape error for DPMultiheadAttention (issue 650) (651)
* Pass kwargs from make_private to _prepare_optimizer (648)
* Fix BatchMemoryManager length (641)
* Fix GPU-CPU device mismatch error in util filter_dilated_rows (633)
* Fix Opacus's runtime error with an empty batch (issue 612) (631)

Bug fixes
* Fix DP MultiheadAttention (598)
* Fix: make prv accountant robust to larger epsilons (606)
* Fix the corner case when the optimizer has no trainable parameters (619)

Highlight: Upgraded to PyTorch 1.13+ as required dependency

New features
* Added clipping schedulers (556)
* Util to check per sample gradients (532)

Bug fixes
* Align DataLoader interface with vanilla PyTorch (543)
* Fix GDP accountant epsilon retrieval changing internal state (541)
* Add option to specify number of steps in UniformSampler (550)
* Fix privacy computation script (565)

New features
* Implement the `PRVAccountant` based on the paper [Numerical Composition of Differential Privacy](https://arxiv.org/abs/2106.02848) (#493)
* Support `nn.EmbeddingBag` (519)

Bug fixes
* Fix benchmarks (503, 507, 508)
* Align `make_private_with_epsilon` with `make_private` (509, 526)
* Test fixes (513, 515, 527, 533)
* Summed discriminator losses to perform one backprop step (474)
* Fixed issue with missing argument in MNIST example (520)
* Functorch gradients: investigation and fix (510)
* Support empty batches (530)