Alpa

Maintenance release.

- Add publications (885)
- Support manual sharding (816)
- Optimize cross-mesh communication (773, 798)
- Support new models: CodeGen, BLOOM-Z (774, 844)
- Add priority scheduling in the model server (852)
- Add guidance on strategy inspection (879)
- Inference stage construction (793, 799)
- Misc bug fixes (876, 878, 873)

- Rebase to jax v0.3.22 (782)
- Support TPU backend for ShardParallel (764)
- Misc improvements for LLM serving

- Fine-tune OPT with 3d parallelism 716
- Support bigscience/bloom 707
- Add model serving controller 720
- Jaxlib update

- Improve OPT generation speed (661)
- Add logprobs endpoint (679)
- Support global norm in pipeline parallelism (672)
- Rebase jax to v0.3.15 (682)
- Improve the stability of the server

- Improve OPT-175B demo https://opt.alpa.ai/. Our website has served 20k+ requests during the last week.
- Add OPT fine-tuning examples https://github.com/alpa-projects/alpa/tree/main/examples/opt_finetune
- Various bug fixes

- Improve OPT-175B demo https://opt.alpa.ai/
- Improve version management
- Various bug fixes