Deepspeed

What's Changed
* Update version.txt after 0.16.4 release by loadams in https://github.com/deepspeedai/DeepSpeed/pull/7063
* fix an outdated doc wrt CUDA_VISIBLE_DEVICES by stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7058
* Tecorigin sdaa accelerator by siqi654321 in https://github.com/deepspeedai/DeepSpeed/pull/6903
* Handle special case of libuv for Windows by loadams in https://github.com/deepspeedai/DeepSpeed/pull/7064
* Bug Fix for offload_states API by U-rara in https://github.com/deepspeedai/DeepSpeed/pull/7050
* Update README with info on newest accelerator by loadams in https://github.com/deepspeedai/DeepSpeed/pull/7065
* Fix TOCTOU issues, switch to fstat by loadams in https://github.com/deepspeedai/DeepSpeed/pull/7067
* config torch to avoid graph breaks caused by logger by ShellyNR in https://github.com/deepspeedai/DeepSpeed/pull/6999
* Fix meta load tensor imcompatible issue by Yejing-Lai in https://github.com/deepspeedai/DeepSpeed/pull/7073
* Replace calls to `python setup.py sdist` with `python -m build --sdist` by loadams in https://github.com/deepspeedai/DeepSpeed/pull/7069
* Revert "Handle special case of libuv for Windows (7064)" by loadams in https://github.com/deepspeedai/DeepSpeed/pull/7076
* Add DeepseekV3 AutoTP. by Yejing-Lai in https://github.com/deepspeedai/DeepSpeed/pull/7045
* Improve inference tutorial docs by loadams in https://github.com/deepspeedai/DeepSpeed/pull/7083
* Pin transformers version on tests that use latest. by loadams in https://github.com/deepspeedai/DeepSpeed/pull/7085
* Update README.md with ICS '23 MoE paper link by siddharth9820 in https://github.com/deepspeedai/DeepSpeed/pull/7087
* Update parallelism for nv-torch-latest/nightly tests due to more GPUs/runner by loadams in https://github.com/deepspeedai/DeepSpeed/pull/7086
* Remove workflows for very old torch versions by loadams in https://github.com/deepspeedai/DeepSpeed/pull/7090
* Use new dlpack api; Formatting fixes by tjruwase in https://github.com/deepspeedai/DeepSpeed/pull/7101
* Avoid graph breaks by disabling sourceless calls in instrument_w_nvtx by deepcharm in https://github.com/deepspeedai/DeepSpeed/pull/7081
* Avoid graph breaks in torch.compile caused by inner classes in the backward hooks by deepcharm in https://github.com/deepspeedai/DeepSpeed/pull/7062
* Only run pre-commit on the changes by hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/7106
* Avoid graph break due to unsupported frozenset by deepcharm in https://github.com/deepspeedai/DeepSpeed/pull/7105
* Fix fused_qkv print model ValueError by Yejing-Lai in https://github.com/deepspeedai/DeepSpeed/pull/7109
* Update references to new X/Twitter handle by loadams in https://github.com/deepspeedai/DeepSpeed/pull/7110
* Update gaudi2 nightly,ci to latest 1.20.0 build by raza-sikander in https://github.com/deepspeedai/DeepSpeed/pull/7093
* fix keep_module_on_host by inkcherry in https://github.com/deepspeedai/DeepSpeed/pull/7112
* Add sequential pytest mark to TestNVMeCheckpointing to resolve pytest forked hangs by loadams in https://github.com/deepspeedai/DeepSpeed/pull/7131
* Training multiple models by tjruwase in https://github.com/deepspeedai/DeepSpeed/pull/7018
* Update CONTRIBUTING.md to reflect changes from CLA to DCO by loadams in https://github.com/deepspeedai/DeepSpeed/pull/7135
* Avoid missing attr error by tjruwase in https://github.com/deepspeedai/DeepSpeed/pull/7133
* Add conditional expression by A-transformer in https://github.com/deepspeedai/DeepSpeed/pull/7119
* Unpin transformers version for most workflows by loadams in https://github.com/deepspeedai/DeepSpeed/pull/7139
* Conditionally quote env vars by saurabhkoshatwar in https://github.com/deepspeedai/DeepSpeed/pull/7071
* Correct the BACKWARD_PREFETCH_SUBMIT mismatch by A-transformer in https://github.com/deepspeedai/DeepSpeed/pull/7120
* Enhance Gaudi2 CI/Nightly Coverage with Model Parallelism and Linear Tests by raza-sikander in https://github.com/deepspeedai/DeepSpeed/pull/7146
* Update container version that runs on A6000 tests. by loadams in https://github.com/deepspeedai/DeepSpeed/pull/7153
* hf tp+zero training doc. by inkcherry in https://github.com/deepspeedai/DeepSpeed/pull/7151
* Avoid graph break by removing redundant requires_grad attr change by deepcharm in https://github.com/deepspeedai/DeepSpeed/pull/7158
* Add destroy to tests to free memory by tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7160
* [NFC] Typo fix in SP layer. by c8ef in https://github.com/deepspeedai/DeepSpeed/pull/7152
* Link AutoTP blog in the front page by hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/7167
* fix `seq_parallel_communication_data_type` constant. by stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7175
* Fix typos in GDS blog by loadams in https://github.com/deepspeedai/DeepSpeed/pull/7177
* Variable batch size and LR scheduler by bm-synth in https://github.com/deepspeedai/DeepSpeed/pull/7104

New Contributors
* siqi654321 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/6903
* A-transformer made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7119
* saurabhkoshatwar made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7071
* c8ef made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7152

**Full Changelog**: https://github.com/deepspeedai/DeepSpeed/compare/v0.16.4...v0.16.5

What's Changed
* Update version.txt after 0.16.3 release by loadams in https://github.com/deepspeedai/DeepSpeed/pull/6965
* Precisely track nvme optimizer offload by tjruwase in https://github.com/deepspeedai/DeepSpeed/pull/6963
* Update build_win.bat script to exclue GDS op as it lacks Windows support. by loadams in https://github.com/deepspeedai/DeepSpeed/pull/6971
* Add CUDA 12.8 support and comment on CUDA 12.7 by loadams in https://github.com/deepspeedai/DeepSpeed/pull/6975
* Update cpu torch latest to use torch 2.6 by loadams in https://github.com/deepspeedai/DeepSpeed/pull/6977
* generalize deepspeed linear and implement it for non cuda systems by oelayan7 in https://github.com/deepspeedai/DeepSpeed/pull/6932
* Update recommended Windows whl building versions by loadams in https://github.com/deepspeedai/DeepSpeed/pull/6983
* Title: Fix setup_env_ranks to Properly Set Environment Variables Instead of Raising Error by fabiosanger in https://github.com/deepspeedai/DeepSpeed/pull/6979
* Specify torchvision in nv-ds-chat workflow (prevents errors with torch 2.6) by loadams in https://github.com/deepspeedai/DeepSpeed/pull/6982
* Remove assumption that padding only occurs on last rank by xylian86 in https://github.com/deepspeedai/DeepSpeed/pull/6974
* Use ds-specific module id to avoid conflicts by tjruwase in https://github.com/deepspeedai/DeepSpeed/pull/6847
* Update A6000 workflows to use newer docker container - 24.09 vs 24.03 by loadams in https://github.com/deepspeedai/DeepSpeed/pull/6967
* Allow NVIDIA Blackwell by fabiendupont in https://github.com/deepspeedai/DeepSpeed/pull/6991
* Update GH org references by tjruwase in https://github.com/deepspeedai/DeepSpeed/pull/6998
* [XPU] max1100 workflow update for docker and softwares by Liangliang-Ma in https://github.com/deepspeedai/DeepSpeed/pull/7003
* autotp training(fix dco) by inkcherry in https://github.com/deepspeedai/DeepSpeed/pull/7004
* import triton files when triton is supported and installed by oelayan7 in https://github.com/deepspeedai/DeepSpeed/pull/6989
* Update A6000 tests transformers version by loadams in https://github.com/deepspeedai/DeepSpeed/pull/7016
* Fix ds-chat CI regression by tjruwase in https://github.com/deepspeedai/DeepSpeed/pull/7015
* [Ulysses tutorial] typos by stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7024
* fix hostname -I for macOS 6497 by fitzjalen in https://github.com/deepspeedai/DeepSpeed/pull/6990
* Update workflows to cuda 12.4 by loadams in https://github.com/deepspeedai/DeepSpeed/pull/7000
* [ROCm] Enable fp_quantizer on ROCm by rraminen in https://github.com/deepspeedai/DeepSpeed/pull/7027
* add gds chinese blog by GuanhuaWang in https://github.com/deepspeedai/DeepSpeed/pull/7034
* Add chinese blog for deepspeed windows, and fix format by hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/7035
* AIO on ROCM by jomayeri in https://github.com/deepspeedai/DeepSpeed/pull/7023
* Control trace cache warnings by tjruwase in https://github.com/deepspeedai/DeepSpeed/pull/7039
* Update CUDA compute capability to support Blackwell by hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/7047
* Update setup.py handling of ROCm cupy by loadams in https://github.com/deepspeedai/DeepSpeed/pull/7051
* nv-ds-chat breaks with latest transformers by loadams in https://github.com/deepspeedai/DeepSpeed/pull/7052
* Rename aio_thread_count to intra_op_parallelism by tjruwase in https://github.com/deepspeedai/DeepSpeed/pull/7056
* add autoTP training zero2 tests by inkcherry in https://github.com/deepspeedai/DeepSpeed/pull/7049
* Fix, bf16 optimizer remove dup loop by wukong1992 in https://github.com/deepspeedai/DeepSpeed/pull/7054

New Contributors
* fabiosanger made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/6979
* fabiendupont made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/6991
* fitzjalen made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/6990
* wukong1992 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7054

**Full Changelog**: https://github.com/deepspeedai/DeepSpeed/compare/v0.16.3...v0.16.4

What's Changed
* Update version.txt after 0.16.2 release by loadams in https://github.com/microsoft/DeepSpeed/pull/6893
* Allow to compile collective for PT>2.3 by NirSonnenschein in https://github.com/microsoft/DeepSpeed/pull/6899
* Zero2: avoid graph breaks in torch.compile by using param_idx by nelyahu in https://github.com/microsoft/DeepSpeed/pull/6803
* hpu_accelerator: use torch.use_deterministic_algorithms by nelyahu in https://github.com/microsoft/DeepSpeed/pull/6897
* Fix error caused by all_reduce call in domino by hwchen2017 in https://github.com/microsoft/DeepSpeed/pull/6880
* Update Gaudi2 jobs to latest 1.19 build by raza-sikander in https://github.com/microsoft/DeepSpeed/pull/6905
* Change compile for pipeline module torch.compile by NirSonnenschein in https://github.com/microsoft/DeepSpeed/pull/6478
* Stage3: Use new torch grad accumulation hooks API by deepcharm in https://github.com/microsoft/DeepSpeed/pull/6773
* Cleanup ops/transformer/inference tests by loadams in https://github.com/microsoft/DeepSpeed/pull/6830
* Fix `checkpointable_layers` Logic by Quentin-Anthony in https://github.com/microsoft/DeepSpeed/pull/6881
* [BUG FIX]:fix get torch.version.cuda error when cuda is None in rocm by hj-wei in https://github.com/microsoft/DeepSpeed/pull/6909
* Add fp8_gemm fallback for non-triton systems by oelayan7 in https://github.com/microsoft/DeepSpeed/pull/6916
* Reduce the device bubble introduced by heavy loop synchronization in coalesced fetch/release(z3_leaf_module) by inkcherry in https://github.com/microsoft/DeepSpeed/pull/6694
* Cleanup ops/transformer/inference tests by loadams in https://github.com/microsoft/DeepSpeed/pull/6925
* Check transformers version in BLOOM for inference v1 by lekurile in https://github.com/microsoft/DeepSpeed/pull/6766
* inference: remove unused _validate_args function by nelyahu in https://github.com/microsoft/DeepSpeed/pull/5505
* Use `torch.log1p` by kit1980 in https://github.com/microsoft/DeepSpeed/pull/6930
* Update python version classifiers by loadams in https://github.com/microsoft/DeepSpeed/pull/6933
* Fix building on Windows with presence of Triton by woct0rdho in https://github.com/microsoft/DeepSpeed/pull/6749
* Fix windows blog examples by loadams in https://github.com/microsoft/DeepSpeed/pull/6934
* Add deepseek autotp by Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/6937
* Add position_ids arg to OPTEmbedding forward function by lekurile in https://github.com/microsoft/DeepSpeed/pull/6939
* Add information on security expectations with this software by loadams in https://github.com/microsoft/DeepSpeed/pull/6941
* Support pure meta model lm_head tp by Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/6812
* Remove op compilation flags due to perf issue by NirSonnenschein in https://github.com/microsoft/DeepSpeed/pull/6944
* Pin nv-a6000 workflow by loadams in https://github.com/microsoft/DeepSpeed/pull/6938
* [inf] Add config var to enable keeping module on host by oelayan7 in https://github.com/microsoft/DeepSpeed/pull/6846
* `warn` to `warning` by qgallouedec in https://github.com/microsoft/DeepSpeed/pull/6952
* Add extra_repr to Linear classes for debugging purpose by Xia-Weiwen in https://github.com/microsoft/DeepSpeed/pull/6954
* Update import for torchvision.transformers by loadams in https://github.com/microsoft/DeepSpeed/pull/6958
* Remove Duplicate Declaration of pandas in `Dockerfile` by Zerohertz in https://github.com/microsoft/DeepSpeed/pull/6959
* Add the missing view operations from sequence parallel(async). by inkcherry in https://github.com/microsoft/DeepSpeed/pull/6750
* Update `torch.norm` to `torch.linalg.norm` and `torch.linalg.vector_norm` by loadams in https://github.com/microsoft/DeepSpeed/pull/6931
* Using explicit GPU upcast for ZeRO-Offload by xylian86 in https://github.com/microsoft/DeepSpeed/pull/6962

New Contributors
* hj-wei made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6909
* kit1980 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6930
* woct0rdho made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6749
* Xia-Weiwen made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6954
* Zerohertz made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6959

**Full Changelog**: https://github.com/microsoft/DeepSpeed/compare/v0.16.2...v0.16.3

What's Changed
* Update pre-commit version by loadams in https://github.com/microsoft/DeepSpeed/pull/6821
* Update version.txt after 0.16.1 release by loadams in https://github.com/microsoft/DeepSpeed/pull/6826
* Pin HPU tests by loadams in https://github.com/microsoft/DeepSpeed/pull/6831
* Flops profiler support einops.einsum by lvhoaa in https://github.com/microsoft/DeepSpeed/pull/6755
* Pin pytest-subtests version for accelerate tests by loadams in https://github.com/microsoft/DeepSpeed/pull/6842
* Inference UTs check for trition support from accelerator by raza-sikander in https://github.com/microsoft/DeepSpeed/pull/6782
* Unpin pytest-subtests now that 0.14.1 is released by loadams in https://github.com/microsoft/DeepSpeed/pull/6844
* Merge LoCo with Zero++ by XingyuXie in https://github.com/microsoft/DeepSpeed/pull/6730
* Fix type error in `ZeROOrderedDict` by oraluben in https://github.com/microsoft/DeepSpeed/pull/6794
* Fix uneven head sequence parallelism bug (6774) by Eugene29 in https://github.com/microsoft/DeepSpeed/pull/6797
* Fix nv-torch-nightly test by pinning transformers by loadams in https://github.com/microsoft/DeepSpeed/pull/6849
* Remove broken links to non-active site by kaiksi-bb in https://github.com/microsoft/DeepSpeed/pull/6854
* Avoid poisoning process with CUDA calls as soon as importing by HollowMan6 in https://github.com/microsoft/DeepSpeed/pull/6810
* Fix xpu tests workflow failure by changing pip index url by Liangliang-Ma in https://github.com/microsoft/DeepSpeed/pull/6864
* Domino updates by GuanhuaWang in https://github.com/microsoft/DeepSpeed/pull/6861
* add domino navigation by GuanhuaWang in https://github.com/microsoft/DeepSpeed/pull/6866
* Update TSC by tjruwase in https://github.com/microsoft/DeepSpeed/pull/6867
* Remove warnings from autodoc and sphinx by loadams in https://github.com/microsoft/DeepSpeed/pull/6788
* Update real_accelerator.py by keiwoo in https://github.com/microsoft/DeepSpeed/pull/6845
* Fix assertion for offloading states by tohtana in https://github.com/microsoft/DeepSpeed/pull/6855
* Remove pin from transformers version and fix Processing/Threading issues in tests by loadams in https://github.com/microsoft/DeepSpeed/pull/6822
* Add MLP/lm_head tp grain size setting. by Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/6828
* Fix --enable_each_rank_log when used with PDSH multi-node runner by akeshet in https://github.com/microsoft/DeepSpeed/pull/6863
* Update transformers ops unit tests to use `requried_torch_version` by loadams in https://github.com/microsoft/DeepSpeed/pull/6884
* Don't error out when cpu accelerator doesn't have torch (as default for whl building) by loadams in https://github.com/microsoft/DeepSpeed/pull/6886
* Add arctic model support by adding w2 to all_reduce by pi314ever in https://github.com/microsoft/DeepSpeed/pull/6856
* Update code owners by tjruwase in https://github.com/microsoft/DeepSpeed/pull/6890

New Contributors
* lvhoaa made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6755
* XingyuXie made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6730
* Eugene29 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6797
* kaiksi-bb made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6854
* HollowMan6 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6810
* keiwoo made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6845
* akeshet made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6863
* pi314ever made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6856

**Full Changelog**: https://github.com/microsoft/DeepSpeed/compare/v0.16.1...v0.16.2

What's Changed
* Update version.txt after 0.16.0 release by loadams in https://github.com/microsoft/DeepSpeed/pull/6786
* Domino news update on readme.md by GuanhuaWang in https://github.com/microsoft/DeepSpeed/pull/6815
* Fix zero checkpoint by xu-song in https://github.com/microsoft/DeepSpeed/pull/6792
* Update python version but now we need to include setuptools on our own by loadams in https://github.com/microsoft/DeepSpeed/pull/6787
* Adding the new feature of FPDT by YJHMITWEB in https://github.com/microsoft/DeepSpeed/pull/6462
* Pin transformers to avoid errors with latest version by loadams in https://github.com/microsoft/DeepSpeed/pull/6820
* Ulyssess offload blog by samadejacobs in https://github.com/microsoft/DeepSpeed/pull/6814
* add FPDT tutorial by samadejacobs in https://github.com/microsoft/DeepSpeed/pull/6813
* Update README.md by samadejacobs in https://github.com/microsoft/DeepSpeed/pull/6824
* Update README.md by samadejacobs in https://github.com/microsoft/DeepSpeed/pull/6825
* Pin transformers version in cpu-torch-latest due to multiprocessing error. by loadams in https://github.com/microsoft/DeepSpeed/pull/6823


**Full Changelog**: https://github.com/microsoft/DeepSpeed/compare/v0.16.0...v0.16.1

What's Changed
* Update version.txt after 0.15.4 release by loadams in https://github.com/microsoft/DeepSpeed/pull/6731
* Update GH hosted workflows to 24.04 by loadams in https://github.com/microsoft/DeepSpeed/pull/6717
* Add COMMITTER file by tjruwase in https://github.com/microsoft/DeepSpeed/pull/6741
* Update AMD apex version by loadams in https://github.com/microsoft/DeepSpeed/pull/6739
* Fix Type Name Inconsistency & Typo in cpu_adam by xylian86 in https://github.com/microsoft/DeepSpeed/pull/6732
* Add Domino code by zhangsmallshark in https://github.com/microsoft/DeepSpeed/pull/6733
* Add data type check for bf16 by hwchen2017 in https://github.com/microsoft/DeepSpeed/pull/6742
* add zero3 module_granularity_threshold to zero optimization. by inkcherry in https://github.com/microsoft/DeepSpeed/pull/6649
* AIO File Offsets by jomayeri in https://github.com/microsoft/DeepSpeed/pull/6641
* Update path for BingBertSquad from DeepSpeedExamples by loadams in https://github.com/microsoft/DeepSpeed/pull/6746
* Sanitize inputs to eval() by loadams in https://github.com/microsoft/DeepSpeed/pull/6745
* Adding the governance doc by minjiazhang in https://github.com/microsoft/DeepSpeed/pull/6748
* Add no_sync context manager by tjruwase in https://github.com/microsoft/DeepSpeed/pull/6675
* Gaudi2 Nightly job for daily check by raza-sikander in https://github.com/microsoft/DeepSpeed/pull/6753
* Disable failing python tests by loadams in https://github.com/microsoft/DeepSpeed/pull/6758
* A faster and more memory-efficient implementation of `zero_to_fp32` by xu-song in https://github.com/microsoft/DeepSpeed/pull/6658
* Pin transformers version to work around latest torch requirements by loadams in https://github.com/microsoft/DeepSpeed/pull/6759
* make xpu ops compatible with oneapi 2025.0 by baodii in https://github.com/microsoft/DeepSpeed/pull/6760
* Add explicit parameters for torch.load by loadams in https://github.com/microsoft/DeepSpeed/pull/6751
* Fix setup.py bash cmd generation to correctly extract git info by nelyahu in https://github.com/microsoft/DeepSpeed/pull/6762
* Use `json_schema_extra` instead of extra keyword in `Field` by qgallouedec in https://github.com/microsoft/DeepSpeed/pull/6764
* Fix potential memory issues when use deepspeed Z3 by wenbinc-Bin in https://github.com/microsoft/DeepSpeed/pull/6726
* Removes unnecessary cloning by swigls in https://github.com/microsoft/DeepSpeed/pull/6761
* Enable torch compile on _allgather_params by deepcharm in https://github.com/microsoft/DeepSpeed/pull/6769
* Unpin with latest transformers fixes by loadams in https://github.com/microsoft/DeepSpeed/pull/6763
* docs: fix HF links by imba-tjd in https://github.com/microsoft/DeepSpeed/pull/6780
* Fix Doc Error: ZeRO Stage 2 gradient partitioning by yewentao256 in https://github.com/microsoft/DeepSpeed/pull/6775
* Cleanup code docs warnings by loadams in https://github.com/microsoft/DeepSpeed/pull/6783
* Domino Blog by GuanhuaWang in https://github.com/microsoft/DeepSpeed/pull/6776
* Update version.txt before release by loadams in https://github.com/microsoft/DeepSpeed/pull/6784
* Revert release workflow by loadams in https://github.com/microsoft/DeepSpeed/pull/6785

New Contributors
* zhangsmallshark made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6733
* hwchen2017 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6742
* minjiazhang made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6748
* qgallouedec made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6764
* wenbinc-Bin made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6726
* swigls made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6761
* imba-tjd made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6780
* yewentao256 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/6775

**Full Changelog**: https://github.com/microsoft/DeepSpeed/compare/v0.15.4...v0.16.0