Opacus

New ways to compute per sample gradients
We're glad to present Opacus v1.2, which contains some major updates to per sample gradient computation mechanisms
and includes all the good stuff from the recent PyTorch releases.
* Functorch - per sample gradients for all
* ExpandedWeights - yet another way to compute per sample gradients
* See [Release notes](https://github.com/pytorch/opacus/releases/tag/v1.2.0)
and [GradSampleModule README](https://github.com/pytorch/opacus/blob/main/opacus/grad_sample/README.md)
for detailed feature explanation

Other improvements
* Fix `utils.unfold2d` with non-symmetric pad/dilation/kernel_size/stride (443)
* Add support for "same" and "valid" padding for hooks-based grad sampler for convolution layers
* Improve model validation to support frozen layers and catch copied parameters (489)
* Remove annoying logging from `set_to_none` (471)
* Improved documentation (480, 478, 482, 485, 486, 487, 488)
* Imtegration test improvements (407, 479, 481. 473)

Bug fixes
* Support layers with a mix of frozen and learnable parameters (437)
* Throw an error when params in optimizer are not the same as that of module's in make_private (439)
* Fix unfold2d and add test (443)

Miscellaneous
* Fix typos in DDP tutorial (438)
* Replace torch einsum with opt_einsum (440)

Bug fixes
* Support tied parameters (417)
* Fix callsite sensitiveness of `zero_grad()` (422, 423)
* Improve microbenchmark argument parsing and tests (425)
* Fix opacus nn.functional import (426)
Miscellaneous
* Add microbenchmarks (412, 416)
* Add more badges to readme (424)

Bug fixes
* Fix accountant when using number of steps instead of epochs
* Add params check when converting BatchNorm to GroupNorm (390)
* Fix typo in gdp accountant mechanism name (386)
* Fix linter errors (392)
* Add friendly and detailed message for unsupported layers (401)
* Run linter on nightly workflow (399)
* Add warning for Gaussian DP accounting (400)
* Clone replacement modules on the same device as original (356)
* Implementing 3D dilation (408)
* fix(batch_memory_manager): Ensures split_idxs use native python types (410)
Miscellaneous
* Migrate nightly CircleCI flows to scheduled pipelines (402)
* Migrate from ubuntu 16.04 to 20.04 on CircleCI (403)

New Feature
* Add support for GDP accounting in get_noise_multiplier (303)
Bug fixes
* Conservative search for target epsilon in get_noise_multiplier (348)
* Warn and ignore "drop_last" when set in DPDataLoader (357)
* Fix per-layer clipping in distributed (347)
Miscellaneous
* Update code of conduct and file headers
* Add "Support Ukraine" banner to opacus website homepage
* Lint fixes

Bug fixes
* DPOptimizer
* Passes through `.defaults` field to match pytorch Optimizer (329)
* Better exception message in `.step()` when p.grad_sample=None (331)
* Correct `closure` call after applying DP noise (330)
* Proper gradient scaling in DDP mode
* Corrections of typos and errors in tutorials
Miscellaneous
* Opacus can be installed with conda: added recipe in conda-forge (326)
* Formatting change in accordance with black-22.1.0