Sglang

Highlights
- Fast JSON Decoding ([blog](https://lmsys.org/blog/2024-02-05-compressed-fsm/))
- Output logprobs for decoding tokens
- Multiple bug fixes

What's Changed
* Fix no-cache mode by Ying1123 in https://github.com/sgl-project/sglang/pull/136
* Support Faster JSON decoding for llava by hnyls2002 in https://github.com/sgl-project/sglang/pull/137
* fix undfined variable by yaya-sy in https://github.com/sgl-project/sglang/pull/142
* jump-forward rename by hnyls2002 in https://github.com/sgl-project/sglang/pull/144
* Add warmup to SRT server by comaniac in https://github.com/sgl-project/sglang/pull/146
* add openai error handler with retry and logger by ChuyueSun in https://github.com/sgl-project/sglang/pull/148
* Temporary fix OpenAI API for Pydantic v1/v2 by comaniac in https://github.com/sgl-project/sglang/pull/153
* Add gptq quantization model support by Arcmoon-Hu in https://github.com/sgl-project/sglang/pull/141
* Support decode token logprobs by comaniac in https://github.com/sgl-project/sglang/pull/130
* Format code & move functions by merrymercy in https://github.com/sgl-project/sglang/pull/155
* [Submodule] Change FlashInfer to import by comaniac in https://github.com/sgl-project/sglang/pull/156
* add `--disable-disk-cache` by hnyls2002 in https://github.com/sgl-project/sglang/pull/160
* Add Auth Token to RuntimeEndpoint by nivibilla in https://github.com/sgl-project/sglang/pull/162
* Fix BaseCache metric by comaniac in https://github.com/sgl-project/sglang/pull/170
* import outlines by hnyls2002 in https://github.com/sgl-project/sglang/pull/168
* Fix token usage with jump forward by comaniac in https://github.com/sgl-project/sglang/pull/174
* Support extra field regex in OpenAI API by comaniac in https://github.com/sgl-project/sglang/pull/172
* Fix the chat template for llava-v1.6-34b & format code by merrymercy in https://github.com/sgl-project/sglang/pull/177
* Update version to 0.1.12 by merrymercy in https://github.com/sgl-project/sglang/pull/178

New Contributors
* yaya-sy made their first contribution in https://github.com/sgl-project/sglang/pull/142
* ChuyueSun made their first contribution in https://github.com/sgl-project/sglang/pull/148
* nivibilla made their first contribution in https://github.com/sgl-project/sglang/pull/162

**Full Changelog**: https://github.com/sgl-project/sglang/compare/v0.1.11...v0.1.12

New Contributors
* isaac-vidas made their first contribution in https://github.com/sgl-project/sglang/pull/80
* Arcmoon-Hu made their first contribution in https://github.com/sgl-project/sglang/pull/75
* CSWellesSun made their first contribution in https://github.com/sgl-project/sglang/pull/84
* haotian-liu made their first contribution in https://github.com/sgl-project/sglang/pull/95
* parasol-aser made their first contribution in https://github.com/sgl-project/sglang/pull/48
* JustinLin610 made their first contribution in https://github.com/sgl-project/sglang/pull/114
* fozziethebeat made their first contribution in https://github.com/sgl-project/sglang/pull/113
* Ja1Zhou made their first contribution in https://github.com/sgl-project/sglang/pull/116

**Full Changelog**: https://github.com/sgl-project/sglang/compare/v0.1.6...v0.1.11

Major features
- Add OpenAI-compatible API server (Completion and ChatCompletion)
- Fix `sgl.select`

All PRs
* Support v1/chat/completions by comaniac in https://github.com/sgl-project/sglang/pull/50
* Fix select and normalized logprobs by merrymercy in https://github.com/sgl-project/sglang/pull/67
* Bump version to 0.1.5 by merrymercy in https://github.com/sgl-project/sglang/pull/33
* Use HTTP link in 3rdparty module by comaniac in https://github.com/sgl-project/sglang/pull/42
* Document sampling parameters by merrymercy in https://github.com/sgl-project/sglang/pull/45
* Increase interpreter parallelism by merrymercy in https://github.com/sgl-project/sglang/pull/46
* Add a llava example by merrymercy in https://github.com/sgl-project/sglang/pull/47
* Support stream=True in v1/completions by comaniac in https://github.com/sgl-project/sglang/pull/49
* Format code & Improve readme by merrymercy in https://github.com/sgl-project/sglang/pull/52
* Fix the possible bug of decode out of memory by hnyls2002 in https://github.com/sgl-project/sglang/pull/36
* Improve error message & Add vicuna template by merrymercy in https://github.com/sgl-project/sglang/pull/57
* Update README.md by eltociear in https://github.com/sgl-project/sglang/pull/58
* Disk FSM cache and adjust code. by hnyls2002 in https://github.com/sgl-project/sglang/pull/63
* Fix select by merrymercy in https://github.com/sgl-project/sglang/pull/64
* Bump version to 0.1.6 by merrymercy in https://github.com/sgl-project/sglang/pull/68

New Contributors
* comaniac made their first contribution in https://github.com/sgl-project/sglang/pull/42
* eltociear made their first contribution in https://github.com/sgl-project/sglang/pull/58

**Full Changelog**: https://github.com/sgl-project/sglang/compare/v0.1.5...v0.1.6

What's Changed
* Fix for T4 GPUs by Ying1123 in https://github.com/sgl-project/sglang/pull/16
* Gemini Backend by caoshiyi in https://github.com/sgl-project/sglang/pull/9
* Teak mem fraction by merrymercy in https://github.com/sgl-project/sglang/pull/20
* Add option to return metadata in async streaming by BabyChouSr in https://github.com/sgl-project/sglang/pull/18
* Expose more arguments to control the scheduling policy by merrymercy in https://github.com/sgl-project/sglang/pull/32
* Rename image_url to image_file by BabyChouSr in https://github.com/sgl-project/sglang/pull/15
* Improve docs by merrymercy in https://github.com/sgl-project/sglang/pull/17
* Improve docs & Rename Gemini -> VertexAI by merrymercy in https://github.com/sgl-project/sglang/pull/19
* Fix streaming by merrymercy in https://github.com/sgl-project/sglang/pull/30

New Contributors
* BabyChouSr made their first contribution in https://github.com/sgl-project/sglang/pull/15
* Ying1123 made their first contribution in https://github.com/sgl-project/sglang/pull/16
* caoshiyi made their first contribution in https://github.com/sgl-project/sglang/pull/9

**Full Changelog**: https://github.com/sgl-project/sglang/compare/v0.1.3...v0.1.5