Deepspeed

What's Changed
* Update version.txt after 0.13.0 release by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4982
* Update FastGen blog title by arashb in https://github.com/microsoft/DeepSpeed/pull/4983
* Fix the MoE-params gradient-scaling by RezaYazdaniAminabadi in https://github.com/microsoft/DeepSpeed/pull/4957
* fix some typo under blogs/ by digger-yu in https://github.com/microsoft/DeepSpeed/pull/4988
* Fix placeholder value in FastGen Blog by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5000
* fix for DS_ENV issue by jeffra in https://github.com/microsoft/DeepSpeed/pull/4992
* Delete unused --deepspeed_mpi command line argument by ShukantPal in https://github.com/microsoft/DeepSpeed/pull/4981
* Make installable without torch by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/5001
* Implement some APIs of HPU accelerator by mmhab in https://github.com/microsoft/DeepSpeed/pull/4935
* Refactor the Qwen positional emebdding config code by ZonePG in https://github.com/microsoft/DeepSpeed/pull/4955


**Full Changelog**: https://github.com/microsoft/DeepSpeed/compare/v0.13.0...v0.13.1

New Features
* [DeepSpeed-FastGen: Introducting Mixtral, Phi-2, and Falcon support with major performance and feature enhancements.](https://github.com/microsoft/DeepSpeed/tree/master/blogs/deepspeed-fastgen/2024-01-19)

What's Changed
* Update version.txt after 0.12.6 release by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4850
* doc corrections by goodship1 in https://github.com/microsoft/DeepSpeed/pull/4861
* Fix exception handling in get_all_ranks_from_group() function by HeyangQin in https://github.com/microsoft/DeepSpeed/pull/4862
* deepspeed engine: fp16 support validation on init by nelyahu in https://github.com/microsoft/DeepSpeed/pull/4843
* Remove hooks on gradient accumulation on engine/optimizer destroy by chiragjn in https://github.com/microsoft/DeepSpeed/pull/4858
* optimize grad_norm calculation in stage3.py by mmhab in https://github.com/microsoft/DeepSpeed/pull/4436
* Fix f-string messages by li-plus in https://github.com/microsoft/DeepSpeed/pull/4865
* [NPU] Fix npu offload bug by CurryRice233 in https://github.com/microsoft/DeepSpeed/pull/4883
* Partition parameters: Minor refactoring of use_secondary_tensor condition by deepcharm in https://github.com/microsoft/DeepSpeed/pull/4868
* Pipeline: Add support to eval micro bs configuration by nelyahu in https://github.com/microsoft/DeepSpeed/pull/4859
* zero_to_fp32.py: Handle a case where shape doesn't have numel attr by nelyahu in https://github.com/microsoft/DeepSpeed/pull/4842
* Add support of Microsoft Phi-2 model to DeepSpeed-FastGen by arashb in https://github.com/microsoft/DeepSpeed/pull/4812
* Support cpu tensors without direct device invocation by abhilash1910 in https://github.com/microsoft/DeepSpeed/pull/3842
* add sharded loading for safetensors in AutoTP by sywangyi in https://github.com/microsoft/DeepSpeed/pull/4854
* [XPU] XPU accelerator support for Intel GPU device by delock in https://github.com/microsoft/DeepSpeed/pull/4547
* enable starcode((kv_head=1)) autotp by Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/4896
* Release overlap_comm & contiguous_gradients restrictions for ZeRO 1 by li-plus in https://github.com/microsoft/DeepSpeed/pull/4887
* [NPU]Add ZeRO-Infinity feature for NPU by misstek in https://github.com/microsoft/DeepSpeed/pull/4809
* fix num_kv_heads sharding in uneven autoTP for Falcon-40b by Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/4712
* Nvme offload checkpoint by eisene in https://github.com/microsoft/DeepSpeed/pull/4707
* Add WarmupCosineLR to Read the Docs by dwyatte in https://github.com/microsoft/DeepSpeed/pull/4916
* Add Habana Labs HPU accelerator support by deepcharm in https://github.com/microsoft/DeepSpeed/pull/4912
* Unit tests for MiCS by zarzen in https://github.com/microsoft/DeepSpeed/pull/4792
* Fix SD workflow to work with latest diffusers version by lekurile in https://github.com/microsoft/DeepSpeed/pull/4918
* [Fix] Fix cpu inference UT failure by delock in https://github.com/microsoft/DeepSpeed/pull/4430
* Add paths to run SD tests by loadams in https://github.com/microsoft/DeepSpeed/pull/4919
* Change PR/schedule triggers for CPU-inference by loadams in https://github.com/microsoft/DeepSpeed/pull/4924
* fix falcon-40b accuracy issue by Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/4895
* Refactor the positional emebdding config code by arashb in https://github.com/microsoft/DeepSpeed/pull/4920
* Pin to triton 2.1.0 to fix issues with nv-inference by loadams in https://github.com/microsoft/DeepSpeed/pull/4929
* Add support of Qwen models (7b, 14b, 72b) to DeepSpeed-FastGen by ZonePG in https://github.com/microsoft/DeepSpeed/pull/4913
* DeepSpeedZeroOptimizer: refactor bit16 flattening to support more accelerators by nelyahu in https://github.com/microsoft/DeepSpeed/pull/4833
* Fix confusing width in simd_load by yzhblind in https://github.com/microsoft/DeepSpeed/pull/4714
* Specify permissions for secrets.GITHUB_TOKEN by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4927
* Enable quantizer op on ROCm by rraminen in https://github.com/microsoft/DeepSpeed/pull/4114
* autoTP for Qwen by inkcherry in https://github.com/microsoft/DeepSpeed/pull/4902
* Allow specifying mii branch for nv-a6000 workflow by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4936
* Only run MII CI for inference changes by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4939
* InfV2 - remove generation config requirement by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4938
* Cache HF model list for inference tests by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4940
* Fix docs inconsistency on default value for `ignore_unused_parameters` by loadams in https://github.com/microsoft/DeepSpeed/pull/4949
* Fix bug in CI model caching by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4951
* fix uneven issue & add balance autotp by Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/4697
* Optimize preprocess for ragged batching by tohtana in https://github.com/microsoft/DeepSpeed/pull/4942
* Fix bug where ZeRO2 never uses the reduce method. by CurryRice233 in https://github.com/microsoft/DeepSpeed/pull/4946
* [docs] Add new autotp supported model in tutorial by delock in https://github.com/microsoft/DeepSpeed/pull/4960
* Add missing op_builder.hpu component for HPU accelerator by nelyahu in https://github.com/microsoft/DeepSpeed/pull/4963
* Stage_1_and_2.py: fix assert for reduce_scatter configurations combinations by nelyahu in https://github.com/microsoft/DeepSpeed/pull/4964
* [MiCS]Add the path to support sequence_data_parallel on MiCS by ys950902 in https://github.com/microsoft/DeepSpeed/pull/4926
* Update the DeepSpeed Phi-2 impl. to work with the HF latest changes by arashb in https://github.com/microsoft/DeepSpeed/pull/4950
* Prevent infinite recursion when DS_ACCELERATOR is set to cuda by ShukantPal in https://github.com/microsoft/DeepSpeed/pull/4962
* Fixes for training models with bf16 + freshly initialized optimizer via `load_module_only` by haileyschoelkopf in https://github.com/microsoft/DeepSpeed/pull/4141
* params partition for skip_init by inkcherry in https://github.com/microsoft/DeepSpeed/pull/4722
* Enhance query APIs for text generation by tohtana in https://github.com/microsoft/DeepSpeed/pull/4965
* Add API to set a module as a leaf node when recursively setting Z3 hooks by tohtana in https://github.com/microsoft/DeepSpeed/pull/4966
* Fix T5 and mistral model meta data error by Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/4958
* FastGen Jan 2024 blog by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4980

New Contributors
* chiragjn made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4858
* li-plus made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4865
* misstek made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4809
* dwyatte made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4916
* ZonePG made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4913
* yzhblind made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4714
* ShukantPal made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4962
* haileyschoelkopf made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4141

**Full Changelog**: https://github.com/microsoft/DeepSpeed/compare/v0.12.6...v0.13.0

What's Changed
* Update version.txt after 0.12.5 release by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4826
* Cache metadata for TP activations and grads by BacharL in https://github.com/microsoft/DeepSpeed/pull/4360
* Inference changes for incorporating meta loading checkpoint by oelayan7 in https://github.com/microsoft/DeepSpeed/pull/4692
* Update CODEOWNERS by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4838
* support baichuan model: by baodii in https://github.com/microsoft/DeepSpeed/pull/4721
* inference engine: check if accelerator supports FP16 by nelyahu in https://github.com/microsoft/DeepSpeed/pull/4832
* Update zeropp.md by goodship1 in https://github.com/microsoft/DeepSpeed/pull/4835
* [NPU] load EXPORT_ENV based on different accelerators to support multi-node training on other devices by minchao-sun in https://github.com/microsoft/DeepSpeed/pull/4830
* Add cuda_accelerator.py to triggers for A6000 test by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4848
* Capture short kernel sequences to graph by inkcherry in https://github.com/microsoft/DeepSpeed/pull/4318
* Checkpointing: Avoid assigning tensor storage with different device by deepcharm in https://github.com/microsoft/DeepSpeed/pull/4836
* engine.py: remove unused _curr_save_path by nelyahu in https://github.com/microsoft/DeepSpeed/pull/4844
* Mixtral FastGen Support by cmikeh2 in https://github.com/microsoft/DeepSpeed/pull/4828

New Contributors
* minchao-sun made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4830

**Full Changelog**: https://github.com/microsoft/DeepSpeed/compare/v0.12.5...v0.12.6

What's Changed
* Fix DS Stable Diffusion for latest diffusers version by lekurile in https://github.com/microsoft/DeepSpeed/pull/4770
* Resolve any '..' in the file paths using os.path.abspath() by rraminen in https://github.com/microsoft/DeepSpeed/pull/4709
* Update dockerfile with updated versions by loadams in https://github.com/microsoft/DeepSpeed/pull/4780
* Run workflows when they are edited by loadams in https://github.com/microsoft/DeepSpeed/pull/4779
* BF16_Optimizer: add support for bf16 grad acc by nelyahu in https://github.com/microsoft/DeepSpeed/pull/4713
* fix autoTP issue for mpt (trust_remote_code=True) by sywangyi in https://github.com/microsoft/DeepSpeed/pull/4787
* Fix Hybrid Engine metrics printing by lekurile in https://github.com/microsoft/DeepSpeed/pull/4789
* [BUG] partition_balanced return wrong result. by zjjMaiMai in https://github.com/microsoft/DeepSpeed/pull/4312
* improve the way to determine whether a variable is None by RUAN-ZX in https://github.com/microsoft/DeepSpeed/pull/4782
* [NPU] Add HcclBackend for 1-bit adam, 1-bit lamb, 0/1 adam by RUAN-ZX in https://github.com/microsoft/DeepSpeed/pull/4733
* Fix for stage3 when setting different communication data type by BacharL in https://github.com/microsoft/DeepSpeed/pull/4540
* Add support of Falcon models (7b, 40b, 180b) to DeepSpeed-FastGen by arashb in https://github.com/microsoft/DeepSpeed/pull/4790
* Switch paths-ignore to single quotes, update paths-ignore on nv-pre-compile-ops by loadams in https://github.com/microsoft/DeepSpeed/pull/4805
* fix for tests using torch<2.1 by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4818
* Universal Checkpoint for Sequence Parallelism by samadejacobs in https://github.com/microsoft/DeepSpeed/pull/4752
* Accelerate CI fix by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4819
* fix [BUG] 'DeepSpeedGPTInference' object has no attribute 'dtype' for… by jxysoft in https://github.com/microsoft/DeepSpeed/pull/4814
* Update broken link in docs by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4822
* Update imports from Transformers by loadams in https://github.com/microsoft/DeepSpeed/pull/4817
* Minor updates to CI workflows by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4823
* fix falcon model load from_config meta_data error by baodii in https://github.com/microsoft/DeepSpeed/pull/4783
* mv DeepSpeedEngine param_names dict init post _configure_distributed_model by nelyahu in https://github.com/microsoft/DeepSpeed/pull/4803
* Refactor launcher user arg parsing by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4824
* Fix 4649 by Alienfeel in https://github.com/microsoft/DeepSpeed/pull/4650

New Contributors
* zjjMaiMai made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4312
* jxysoft made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4814
* baodii made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4783
* Alienfeel made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4650

**Full Changelog**: https://github.com/microsoft/DeepSpeed/compare/v0.12.4...v0.12.5

What's Changed
* Update version.txt after 0.12.3 release by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4673
* [MII] catch error wrt HF version and Mistral by jeffra in https://github.com/microsoft/DeepSpeed/pull/4634
* [NPU] Add NPU support for unit test by RUAN-ZX in https://github.com/microsoft/DeepSpeed/pull/4569
* [op-builder] use unique exceptions for cuda issues by jeffra in https://github.com/microsoft/DeepSpeed/pull/4653
* Add stable diffusion unit test by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/2496
* [CANN] Support cpu offload optimizer for Ascend NPU by hipudding in https://github.com/microsoft/DeepSpeed/pull/4568
* Inference Checkpoints in V2 by cmikeh2 in https://github.com/microsoft/DeepSpeed/pull/4664
* KV Cache Improved Flexibility by cmikeh2 in https://github.com/microsoft/DeepSpeed/pull/4668
* Fix for when prompt contains an odd num of apostrophes by oelayan7 in https://github.com/microsoft/DeepSpeed/pull/4660
* universal-ckp: support megatron-deepspeed llama model by mosheisland in https://github.com/microsoft/DeepSpeed/pull/4666
* Add new MII unit tests by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4693
* [Bug fix] WarmupCosineLR issues by sbwww in https://github.com/microsoft/DeepSpeed/pull/4688
* infV2 fix for OPT size variants by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4694
* Add get and set APIs for the ZeRO-3 partitioned parameters by yiliu30 in https://github.com/microsoft/DeepSpeed/pull/4681
* Remove unneeded dict reinit (fix for 4565) by eisene in https://github.com/microsoft/DeepSpeed/pull/4702
* Update flops profiler to recurse by loadams in https://github.com/microsoft/DeepSpeed/pull/4374
* Communication Optimization for Large-Scale Training by RezaYazdaniAminabadi in https://github.com/microsoft/DeepSpeed/pull/4695
* [docs] Intel inference blog by jeffra in https://github.com/microsoft/DeepSpeed/pull/4734
* use all_gather_into_tensor instead of all_gather by taozhiwei in https://github.com/microsoft/DeepSpeed/pull/4705
* Install `deepspeed-kernels` only on Linux by aphedges in https://github.com/microsoft/DeepSpeed/pull/4739
* Add nv-sd badge to README by loadams in https://github.com/microsoft/DeepSpeed/pull/4747
* Re-organize `.gitignore` file to be parsed properly by aphedges in https://github.com/microsoft/DeepSpeed/pull/4740
* fix mics run with offload++ by GuanhuaWang in https://github.com/microsoft/DeepSpeed/pull/4749
* Fix logger formatting for partitioning flags by OAfzal in https://github.com/microsoft/DeepSpeed/pull/4728
* fix: to solve 4726 by RUAN-ZX in https://github.com/microsoft/DeepSpeed/pull/4727
* Add safetensors support by jihnenglin in https://github.com/microsoft/DeepSpeed/pull/4659

New Contributors
* RUAN-ZX made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4569
* oelayan7 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4660
* sbwww made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4688
* yiliu30 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4681
* eisene made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4702
* taozhiwei made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4705
* OAfzal made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4728
* jihnenglin made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4659

**Full Changelog**: https://github.com/microsoft/DeepSpeed/compare/v0.12.3...v0.12.4

New Bug Fixes
* Stable Diffusion now supported with latest Torch, diffusers, and Triton versions.

What's Changed
* Update version.txt after 0.12.2 release by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4617
* Fix figure in FlexGen blog by tohtana in https://github.com/microsoft/DeepSpeed/pull/4624
* Fix figure of llama2 13B in DS-FlexGen blog by tohtana in https://github.com/microsoft/DeepSpeed/pull/4625
* Fix config format by xu-song in https://github.com/microsoft/DeepSpeed/pull/4594
* Guanhua/partial offload rebase v2 (590) by GuanhuaWang in https://github.com/microsoft/DeepSpeed/pull/4636
* offload++ blog (623) by GuanhuaWang in https://github.com/microsoft/DeepSpeed/pull/4637
* Update README in offloadpp blog by GuanhuaWang in https://github.com/microsoft/DeepSpeed/pull/4641
* [docs] update news items by jeffra in https://github.com/microsoft/DeepSpeed/pull/4640
* DeepSpeed-FastGen Chinese Blog by HeyangQin in https://github.com/microsoft/DeepSpeed/pull/4642
* Fix issues with torch cpu builds by loadams in https://github.com/microsoft/DeepSpeed/pull/4639
* Isolate src code and testing for DeepSpeed-FastGen by cmikeh2 in https://github.com/microsoft/DeepSpeed/pull/4610
* Add Japanese blog for DeepSpeed-FastGen by tohtana in https://github.com/microsoft/DeepSpeed/pull/4651
* Fix for MII unit tests by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4652
* Enhance the robustness of `module_state_dict` by LZHgrla in https://github.com/microsoft/DeepSpeed/pull/4587
* Enable ZeRO3 allgather for multiple dtypes by tohtana in https://github.com/microsoft/DeepSpeed/pull/4647
* add option to disable pipeline partitioning by nelyahu in https://github.com/microsoft/DeepSpeed/pull/4322
* Added __HIP_PLATFORM_AMD__=1 for non JIT build by rraminen in https://github.com/microsoft/DeepSpeed/pull/4585
* Fix rope_theta arg for diffusers_attention by lekurile in https://github.com/microsoft/DeepSpeed/pull/4656
* tl.dot(a,b, trans_b=True) is not supported by triton2.0+ , updating this api by bmedishe in https://github.com/microsoft/DeepSpeed/pull/4541
* Update ds-chat workflow to work w/ deepspeed-chat install by lekurile in https://github.com/microsoft/DeepSpeed/pull/4598
* Diffusers attention script update triton2.1 by bmedishe in https://github.com/microsoft/DeepSpeed/pull/4573
* Fix the openfold training. by cctry in https://github.com/microsoft/DeepSpeed/pull/4657
* Universal ckp fixes by mosheisland in https://github.com/microsoft/DeepSpeed/pull/4588
* Update .gitignore [Adding comments , Improved documentation] by Nadav23AnT in https://github.com/microsoft/DeepSpeed/pull/4631
* Update lr_schedules.py by CoinCheung in https://github.com/microsoft/DeepSpeed/pull/4563
* Fix UNET and VAE implementations for new diffusers version by lekurile in https://github.com/microsoft/DeepSpeed/pull/4663
* fix num_kv_heads sharding in autoTP for the new in-repo Falcon-40B by dc3671 in https://github.com/microsoft/DeepSpeed/pull/4654

New Contributors
* xu-song made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4594
* LZHgrla made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4587
* mosheisland made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4588
* Nadav23AnT made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4631
* CoinCheung made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4563

**Full Changelog**: https://github.com/microsoft/DeepSpeed/compare/v0.12.2...v0.12.3