Deepspeed

What's Changed
* Quick bug fix direct to `master` to ensure mismatched cuda environments are shown to the user https://github.com/microsoft/DeepSpeed/commit/4f7dd7214b1d81dbbdff826015a67accc10390d2
* Update version.txt after 0.12.1 release by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4615


**Full Changelog**: https://github.com/microsoft/DeepSpeed/compare/v0.12.1...v0.12.2

What's Changed
* Update version.txt after 0.12.0 release by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4611
* Add number for latency comparison by tohtana in https://github.com/microsoft/DeepSpeed/pull/4612
* Update minor CUDA version compatibility. by cmikeh2 in https://github.com/microsoft/DeepSpeed/pull/4613


**Full Changelog**: https://github.com/microsoft/DeepSpeed/compare/v0.12.0...v0.12.1

New features
* [DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference](https://github.com/microsoft/DeepSpeed/tree/master/blogs/deepspeed-fastgen)

What's Changed
* Update version.txt after 0.11.2 release by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4609
* Pin transformers in nv-inference by loadams in https://github.com/microsoft/DeepSpeed/pull/4606
* DeepSpeed-FastGen by cmikeh2 in https://github.com/microsoft/DeepSpeed/pull/4604
* DeepSpeed-FastGen blog by jeffra in https://github.com/microsoft/DeepSpeed/pull/4607


**Full Changelog**: https://github.com/microsoft/DeepSpeed/compare/v0.11.2...v0.12.0

What's Changed
* Update version.txt after 0.11.1 release by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4484
* Update DS_BUILD_* references. by loadams in https://github.com/microsoft/DeepSpeed/pull/4485
* Introduce pydantic_v1 compatibility module for pydantic>=2.0.0 support by ringohoffman in https://github.com/microsoft/DeepSpeed/pull/4407
* Enable control over timeout with environment variable by BramVanroy in https://github.com/microsoft/DeepSpeed/pull/4405
* Update ROCm verison by loadams in https://github.com/microsoft/DeepSpeed/pull/4486
* adding 8bit dequantization kernel for asym fine-grained block quantization in zero-inference by stephen-youn in https://github.com/microsoft/DeepSpeed/pull/4450
* Fix scale factor on flops profiler by loadams in https://github.com/microsoft/DeepSpeed/pull/4500
* add DeepSpeed4Science white paper by conglongli in https://github.com/microsoft/DeepSpeed/pull/4502
* [CCLBackend] update API by Liangliang-Ma in https://github.com/microsoft/DeepSpeed/pull/4378
* Ulysses: add col-ai evaluation by samadejacobs in https://github.com/microsoft/DeepSpeed/pull/4517
* Ulysses: Update README.md by samadejacobs in https://github.com/microsoft/DeepSpeed/pull/4518
* add available memory check to accelerators by jeffra in https://github.com/microsoft/DeepSpeed/pull/4508
* clear redundant parameters in zero3 bwd hook by inkcherry in https://github.com/microsoft/DeepSpeed/pull/4520
* Add NPU FusedAdam support by CurryRice233 in https://github.com/microsoft/DeepSpeed/pull/4343
* fix error type issue in deepspeed/comm/ccl.py by Liangliang-Ma in https://github.com/microsoft/DeepSpeed/pull/4521
* Fixed deepspeed.comm.monitored_barrier call by Quentin-Anthony in https://github.com/microsoft/DeepSpeed/pull/4496
* [Bug fix] Add rope_theta for llama config by cupertank in https://github.com/microsoft/DeepSpeed/pull/4480
* [ROCm] Add rocblas header by rraminen in https://github.com/microsoft/DeepSpeed/pull/4538
* [docs] ZeRO infinity slides and blog by jeffra in https://github.com/microsoft/DeepSpeed/pull/4542
* Switch from HIP_PLATFORM_HCC to HIP_PLATFORM_AMD by loadams in https://github.com/microsoft/DeepSpeed/pull/4539
* Turn off I_MPI_PIN for impi launcher by delock in https://github.com/microsoft/DeepSpeed/pull/4531
* [docs] paper updates by jeffra in https://github.com/microsoft/DeepSpeed/pull/4543
* ROCm 6.0 prep changes by loadams in https://github.com/microsoft/DeepSpeed/pull/4537
* Fix RTD builds by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4558
* pipe engine _aggregate_total_loss: more efficient loss concatenation by nelyahu in https://github.com/microsoft/DeepSpeed/pull/4327
* Add missing rocblas include by loadams in https://github.com/microsoft/DeepSpeed/pull/4557
* Enable universal checkpoint for zero stage 1 by tjruwase in https://github.com/microsoft/DeepSpeed/pull/4516
* [AutoTP] Make AutoTP work when num_heads not divisible by number of workers by delock in https://github.com/microsoft/DeepSpeed/pull/4011
* Fix the sequence-parallelism for the dense model architecture by RezaYazdaniAminabadi in https://github.com/microsoft/DeepSpeed/pull/4530
* engine.py - save_checkpoint: only rank-0 should create the save dir by nelyahu in https://github.com/microsoft/DeepSpeed/pull/4536
* Remove PP Grad Tail Check by Quentin-Anthony in https://github.com/microsoft/DeepSpeed/pull/2538
* Added __HIP_PLATFORM_AMD__=1 by rraminen in https://github.com/microsoft/DeepSpeed/pull/4570
* fix multiple definition while building evoformer by fecet in https://github.com/microsoft/DeepSpeed/pull/4556
* Don't check overflow for bf16 data type by hablb in https://github.com/microsoft/DeepSpeed/pull/4512
* Public update by yaozhewei in https://github.com/microsoft/DeepSpeed/pull/4583
* [docs] paper updates by jeffra in https://github.com/microsoft/DeepSpeed/pull/4584
* Disable CPU inference on PRs by loadams in https://github.com/microsoft/DeepSpeed/pull/4590

New Contributors
* ringohoffman made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4407
* BramVanroy made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4405
* cupertank made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4480

**Full Changelog**: https://github.com/microsoft/DeepSpeed/compare/v0.11.1...v0.11.2

What's Changed
* Fix bug in bfloat16 optimizer related to checkpointing by okoge-kaz in https://github.com/microsoft/DeepSpeed/pull/4434
* Move tensors to device if mp is not enabled by deepcharm in https://github.com/microsoft/DeepSpeed/pull/4461
* Fix torch import causing release build failure by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4468
* add lm_head and embed_out tensor parallel by Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/3962
* Fix release workflow by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4483

New Contributors
* okoge-kaz made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4434
* deepcharm made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4461

**Full Changelog**: https://github.com/microsoft/DeepSpeed/compare/v0.11.0...v0.11.1

New features
* [DeepSpeed-VisualChat: Improve Your Chat Experience with Multi-Round Multi-Image Inputs](https://github.com/microsoft/DeepSpeed/tree/master/blogs/deepspeed-visualchat/10-03-2023/README.md) [[English](https://github.com/microsoft/DeepSpeed/tree/master/blogs/deepspeed-visualchat/10-03-2023/README.md)] [[中文](https://github.com/microsoft/DeepSpeed/blob/master/blogs/deepspeed-visualchat/10-03-2023/README-Chinese.md)] [[日本語](https://github.com/microsoft/DeepSpeed/blob/master/blogs/deepspeed-visualchat/10-03-2023/README-Japanese.md)]
* Announcing the DeepSpeed4Science Initiative: Enabling large-scale scientific discovery through sophisticated AI system technologies [[DeepSpeed4Science website](https://deepspeed4science.ai/)] [[Tutorials](https://www.deepspeed.ai/deepspeed4science/)] [[Blog](https://www.microsoft.com/en-us/research/blog/announcing-the-deepspeed4science-initiative-enabling-large-scale-scientific-discovery-through-sophisticated-ai-system-technologies/)] [[中文](https://github.com/microsoft/DeepSpeed/blob/master/blogs/deepspeed4science/chinese/README.md)] [[日本語](https://github.com/microsoft/DeepSpeed/blob/master/blogs/deepspeed4science/japanese/README.md)]

What's Changed
* added a model check for use_triton in deepspeed by stephen-youn in https://github.com/microsoft/DeepSpeed/pull/4266
* Update release and bump patch versioning flow by loadams in https://github.com/microsoft/DeepSpeed/pull/4286
* README update by tjruwase in https://github.com/microsoft/DeepSpeed/pull/4303
* Update README.md by NinoRisteski in https://github.com/microsoft/DeepSpeed/pull/4316
* Handle empty parameter groups by tjruwase in https://github.com/microsoft/DeepSpeed/pull/4277
* Clean up modeling code by loadams in https://github.com/microsoft/DeepSpeed/pull/4320
* Fix Zero3 contiguous grads, reduce scatter false accuracy issue by nelyahu in https://github.com/microsoft/DeepSpeed/pull/4321
* Add release version checking by loadams in https://github.com/microsoft/DeepSpeed/pull/4328
* clear redundant timers by starkhu in https://github.com/microsoft/DeepSpeed/pull/4308
* DS-Chat BLOOM: Fix Attention mask by lekurile in https://github.com/microsoft/DeepSpeed/pull/4338
* Fix a bug in the implementation of dequantization for inference by sakogan in https://github.com/microsoft/DeepSpeed/pull/3433
* Suppress noise by tjruwase in https://github.com/microsoft/DeepSpeed/pull/4310
* Fix skipped inference tests by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4336
* Fix autotune to support Triton 2.1 by stephen-youn in https://github.com/microsoft/DeepSpeed/pull/4340
* Pass base_dir to model files can be loaded for auto-tp/meta-tensor. by awan-10 in https://github.com/microsoft/DeepSpeed/pull/4348
* Support InternLM by wangruohui in https://github.com/microsoft/DeepSpeed/pull/4137
* DeepSpeed4Science by conglongli in https://github.com/microsoft/DeepSpeed/pull/4357
* fix deepspeed4science links by conglongli in https://github.com/microsoft/DeepSpeed/pull/4358
* Add the policy to run llama model from the official repo by RezaYazdaniAminabadi in https://github.com/microsoft/DeepSpeed/pull/4313
* Check inference input_id tokens length by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4349
* add deepspeed4science blog link by conglongli in https://github.com/microsoft/DeepSpeed/pull/4364
* Update conda env to have max pydantic version by loadams in https://github.com/microsoft/DeepSpeed/pull/4362
* Enable workflow dispatch on Torch 1.10 CI tests by loadams in https://github.com/microsoft/DeepSpeed/pull/4361
* deepspeed4science chinese blog by conglongli in https://github.com/microsoft/DeepSpeed/pull/4366
* deepspeed4science japanese blog by conglongli in https://github.com/microsoft/DeepSpeed/pull/4369
* Openfold fix by cctry in https://github.com/microsoft/DeepSpeed/pull/4368
* [BUG] add the missing method to MPS accelerator by cli99 in https://github.com/microsoft/DeepSpeed/pull/4363
* Fix multinode runner to properly append to PDSH_SSH_ARGS_APPEND by loadams in https://github.com/microsoft/DeepSpeed/pull/4373
* Fix min torch version by tjruwase in https://github.com/microsoft/DeepSpeed/pull/4375
* Fix llama meta tensor loading in AutoTP and kernel injected inference by zeyugao in https://github.com/microsoft/DeepSpeed/pull/3608
* adds triton flash attention2 kernel by stephen-youn in https://github.com/microsoft/DeepSpeed/pull/4337
* Allow multiple inference engines in single script by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4384
* Save/restore step in param groups with zero 1 or 2 by tohtana in https://github.com/microsoft/DeepSpeed/pull/4396
* Fix incorrect assignment of self.quantized_nontrainable_weights by VeryLazyBoy in https://github.com/microsoft/DeepSpeed/pull/4399
* update deepspeed4science blog by conglongli in https://github.com/microsoft/DeepSpeed/pull/4408
* Add torch no grad condition by ajindal1 in https://github.com/microsoft/DeepSpeed/pull/4391
* Update nv-transformers workflow to use cu11.6 by loadams in https://github.com/microsoft/DeepSpeed/pull/4412
* Add condition when dimension is greater than 2 by ajindal1 in https://github.com/microsoft/DeepSpeed/pull/4390
* [CPU] Add CPU AutoTP UT. by Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/4263
* fix cpu loading model partition OOM by Yejing-Lai in https://github.com/microsoft/DeepSpeed/pull/4353
* Update cpu_inference checkout action by loadams in https://github.com/microsoft/DeepSpeed/pull/4424
* Zero infinity xpu support by Liangliang-Ma in https://github.com/microsoft/DeepSpeed/pull/4130
* [CCLBackend] Using parallel memcpy for inference_all_reduce by delock in https://github.com/microsoft/DeepSpeed/pull/4404
* Change default `set_to_none=true` in `zero_grad` methods by Jackmin801 in https://github.com/microsoft/DeepSpeed/pull/4438
* Small docstring fix by Jackmin801 in https://github.com/microsoft/DeepSpeed/pull/4431
* fix: check-license by Jackmin801 in https://github.com/microsoft/DeepSpeed/pull/4432
* Fixup check release version script by loadams in https://github.com/microsoft/DeepSpeed/pull/4413
* Enable ad-hoc running of cpu_inference by loadams in https://github.com/microsoft/DeepSpeed/pull/4444
* Fix wrong documentation of `ignore_unused_parameters` by UniverseFly in https://github.com/microsoft/DeepSpeed/pull/4418
* DeepSpeed-VisualChat Blog by xiaoxiawu-microsoft in https://github.com/microsoft/DeepSpeed/pull/4446
* Fix a bug in DeepSpeedMLP by sakogan in https://github.com/microsoft/DeepSpeed/pull/4389
* documenting load_from_fp32_weights config parameter by clumsy in https://github.com/microsoft/DeepSpeed/pull/4449
* Add Japanese translation of DS-VisualChat blog by tohtana in https://github.com/microsoft/DeepSpeed/pull/4454
* fix blog format by conglongli in https://github.com/microsoft/DeepSpeed/pull/4456
* Update README-Japanese.md by conglongli in https://github.com/microsoft/DeepSpeed/pull/4457
* DeepSpeed-VisualChat Chinese blog by conglongli in https://github.com/microsoft/DeepSpeed/pull/4458
* CI fix for torch 2.1 release by mrwyattii in https://github.com/microsoft/DeepSpeed/pull/4452
* fix lm head overriden issue, move it from checkpoint in-loop loading … by sywangyi in https://github.com/microsoft/DeepSpeed/pull/4206
* feat: add Lion by enneamer in https://github.com/microsoft/DeepSpeed/pull/4331
* pipe engine eval_batch: add option to disable loss broadcast by nelyahu in https://github.com/microsoft/DeepSpeed/pull/4326
* Add release flow by loadams in https://github.com/microsoft/DeepSpeed/pull/4467

New Contributors
* nelyahu made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4321
* starkhu made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4308
* sakogan made their first contribution in https://github.com/microsoft/DeepSpeed/pull/3433
* cctry made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4368
* zeyugao made their first contribution in https://github.com/microsoft/DeepSpeed/pull/3608
* VeryLazyBoy made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4399
* ajindal1 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4391
* Liangliang-Ma made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4130
* Jackmin801 made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4438
* UniverseFly made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4418
* enneamer made their first contribution in https://github.com/microsoft/DeepSpeed/pull/4331

**Full Changelog**: https://github.com/microsoft/DeepSpeed/compare/v0.10.3...v0.11.0