Deepspeed

What's Changed
* GatheredParameters - accept a tuple of params by stas00 in https://github.com/microsoft/DeepSpeed/pull/1941
* Update partition_parameters.py by manuelciosici in https://github.com/microsoft/DeepSpeed/pull/1943
* fix step in adam by szhengac in https://github.com/microsoft/DeepSpeed/pull/1823
* [pipe] prevent deadlock with multiple evals sequence by stas00 in https://github.com/microsoft/DeepSpeed/pull/1944
* Fairseq support by jeffra in https://github.com/microsoft/DeepSpeed/pull/1915
* DeepSpeed needs to start cleaning up by tjruwase in https://github.com/microsoft/DeepSpeed/pull/1947
* trivial fix by kisseternity in https://github.com/microsoft/DeepSpeed/pull/1954
* Enabling CUDA-graph for the bert-type models by RezaYazdaniAminabadi in https://github.com/microsoft/DeepSpeed/pull/1952
* Add loss scale guard to avoid inf loop by Quentin-Anthony in https://github.com/microsoft/DeepSpeed/pull/1958
* [launcher] add option to bypass ssh check by liamcli in https://github.com/microsoft/DeepSpeed/pull/1957
* Bump nokogiri from 1.13.4 to 1.13.6 in /docs by dependabot in https://github.com/microsoft/DeepSpeed/pull/1965
* Fix typo in timer.py by Quentin-Anthony in https://github.com/microsoft/DeepSpeed/pull/1964
* [docs] fix dependabot version issue by jeffra in https://github.com/microsoft/DeepSpeed/pull/1966
* Don't add curand on rocm by jeffra in https://github.com/microsoft/DeepSpeed/pull/1968
* Add Unidirectional Sparse Attention Type to BigBird and BSLongformer by Quentin-Anthony in https://github.com/microsoft/DeepSpeed/pull/1959
* Fix: Sparse tensors not updating by Dipet in https://github.com/microsoft/DeepSpeed/pull/1914
* Fixing several bugs in the inference-api and the kernels by RezaYazdaniAminabadi in https://github.com/microsoft/DeepSpeed/pull/1951

New Contributors
* Quentin-Anthony made their first contribution in https://github.com/microsoft/DeepSpeed/pull/1958

**Full Changelog**: https://github.com/microsoft/DeepSpeed/compare/v0.6.4...v0.6.5

What's Changed
* [fix] Windows installs cannot import fcntl by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/1921
* [build] explicitly add op_builder to manifest by jeffra in https://github.com/microsoft/DeepSpeed/pull/1920
* Enable DeepSpeed inference on ROCm by rraminen in https://github.com/microsoft/DeepSpeed/pull/1922
* bf16 inference by tjruwase in https://github.com/microsoft/DeepSpeed/pull/1917
* spell err by kisseternity in https://github.com/microsoft/DeepSpeed/pull/1929
* [ZeRO-3] Rename confusing log message by jeffra in https://github.com/microsoft/DeepSpeed/pull/1932
* [bug] Fix time log error in PipelineEngine by Codle in https://github.com/microsoft/DeepSpeed/pull/1934
* Improve z3 trace management by tjruwase in https://github.com/microsoft/DeepSpeed/pull/1916

New Contributors
* kisseternity made their first contribution in https://github.com/microsoft/DeepSpeed/pull/1929
* Codle made their first contribution in https://github.com/microsoft/DeepSpeed/pull/1934

**Full Changelog**: https://github.com/microsoft/DeepSpeed/compare/v0.6.3...v0.6.4

What's Changed
* Fix setup.py crash when torch is not installed. by PaperclipBadger in https://github.com/microsoft/DeepSpeed/pull/1866
* Add support for AWS SageMaker. by matherit in https://github.com/microsoft/DeepSpeed/pull/1868
* Fix broken links by tjruwase in https://github.com/microsoft/DeepSpeed/pull/1873
* [docs] add amd blog to website by jeffra in https://github.com/microsoft/DeepSpeed/pull/1874
* [docs] add moe paper by jeffra in https://github.com/microsoft/DeepSpeed/pull/1875
* Supporting multiple modules injection with a single policy when they … by samyam in https://github.com/microsoft/DeepSpeed/pull/1869
* [docs] fix dead links by jeffra in https://github.com/microsoft/DeepSpeed/pull/1877
* add now required `-lcurand` to solve `undefined symbol: curandCreateGenerator` by stas00 in https://github.com/microsoft/DeepSpeed/pull/1879
* Bug fix for flops profilers output by VisionTheta in https://github.com/microsoft/DeepSpeed/pull/1885
* Bump nokogiri from 1.13.3 to 1.13.4 in /docs by dependabot in https://github.com/microsoft/DeepSpeed/pull/1889
* [docs] fix commonmarker security issue by jeffra in https://github.com/microsoft/DeepSpeed/pull/1892
* bf16+pipeline parallelism by tjruwase in https://github.com/microsoft/DeepSpeed/pull/1801
* fix file ordering by szhengac in https://github.com/microsoft/DeepSpeed/pull/1822
* Use f-strings where possible by manuelciosici in https://github.com/microsoft/DeepSpeed/pull/1900
* [partition_parameters.py] better diagnostics by stas00 in https://github.com/microsoft/DeepSpeed/pull/1887
* comm backend: cast bool when not supported by torch2cupy by conglongli in https://github.com/microsoft/DeepSpeed/pull/1894
* Use cuda events to improve timing for multi-stream execution by tjruwase in https://github.com/microsoft/DeepSpeed/pull/1881
* Fix multiple zero 3 tracing errors by tjruwase in https://github.com/microsoft/DeepSpeed/pull/1901
* Improve ds_report output for HIP/ROCm by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/1906
* Fix launcher for reading env vars by szhengac in https://github.com/microsoft/DeepSpeed/pull/1907
* Fix OOM and type mismatch by tjruwase in https://github.com/microsoft/DeepSpeed/pull/1884

New Contributors
* PaperclipBadger made their first contribution in https://github.com/microsoft/DeepSpeed/pull/1866
* matherit made their first contribution in https://github.com/microsoft/DeepSpeed/pull/1868
* VisionTheta made their first contribution in https://github.com/microsoft/DeepSpeed/pull/1885
* szhengac made their first contribution in https://github.com/microsoft/DeepSpeed/pull/1822

Misc

**Full Changelog**: https://github.com/microsoft/DeepSpeed/compare/v0.6.1...v0.6.3

Release notes

New features

* [Advancing MoE inference and training to power next-generation AI scale](https://www.microsoft.com/en-us/research/blog/deepspeed-advancing-moe-inference-and-training-to-power-next-generation-ai-scale/)
* MoE inference
* PR-MoE model support
* AMD support (1430)
* Various ZeRO Stage3 Optimizations + Improvements (1453)

Special thanks to our contributors in this release
stas00, jithunnair-amd, rraminen, jeffdaily, okakarpa, jfc4050, raamjad, aphedges, SeanNaren, liamcli, andriyor, manuelciosici