Openllm

Usage

All available models: openllm models

To start a LLM: python -m openllm start HuggingFaceH4/zephyr-7b-beta

To run OpenLLM within a container environment (requires GPUs): docker run --gpus all -it -P -v $PWD/data:$HOME/.cache/huggingface/ ghcr.io/bentoml/openllm:0.4.29 start HuggingFaceH4/zephyr-7b-beta

Find more information about this release in the [CHANGELOG.md](https://github.com/bentoml/OpenLLM/blob/main/CHANGELOG.md)



**Full Changelog**: https://github.com/bentoml/OpenLLM/compare/v0.4.28...v0.4.29

Usage

All available models: openllm models

To start a LLM: python -m openllm start HuggingFaceH4/zephyr-7b-beta

To run OpenLLM within a container environment (requires GPUs): docker run --gpus all -it -P -v $PWD/data:$HOME/.cache/huggingface/ ghcr.io/bentoml/openllm:0.4.28 start HuggingFaceH4/zephyr-7b-beta

Find more information about this release in the [CHANGELOG.md](https://github.com/bentoml/OpenLLM/blob/main/CHANGELOG.md)



What's Changed
* fix(baichuan): supported from baichuan 2 from now on. by MingLiangDai in https://github.com/bentoml/OpenLLM/pull/728


**Full Changelog**: https://github.com/bentoml/OpenLLM/compare/v0.4.27...v0.4.28

Usage

All available models: openllm models

To start a LLM: python -m openllm start HuggingFaceH4/zephyr-7b-beta

To run OpenLLM within a container environment (requires GPUs): docker run --gpus all -it -P -v $PWD/data:$HOME/.cache/huggingface/ ghcr.io/bentoml/openllm:0.4.26 start HuggingFaceH4/zephyr-7b-beta

Find more information about this release in the [CHANGELOG.md](https://github.com/bentoml/OpenLLM/blob/main/CHANGELOG.md)



What's Changed
* fix(infra): setup higher timer for building container images by aarnphm in https://github.com/bentoml/OpenLLM/pull/723
* fix(client): correct schemas parser from correct response output by aarnphm in https://github.com/bentoml/OpenLLM/pull/724
* feat(openai): chat templates and complete control of prompt generation by aarnphm in https://github.com/bentoml/OpenLLM/pull/725


**Full Changelog**: https://github.com/bentoml/OpenLLM/compare/v0.4.25...v0.4.26

Usage

All available models: openllm models

To start a LLM: python -m openllm start HuggingFaceH4/zephyr-7b-beta

To run OpenLLM within a container environment (requires GPUs): docker run --gpus all -it -P -v $PWD/data:$HOME/.cache/huggingface/ ghcr.io/bentoml/openllm:0.4.25 start HuggingFaceH4/zephyr-7b-beta

Find more information about this release in the [CHANGELOG.md](https://github.com/bentoml/OpenLLM/blob/main/CHANGELOG.md)



What's Changed
* fix(openai): correct stop tokens and finish_reason state by aarnphm in https://github.com/bentoml/OpenLLM/pull/722


**Full Changelog**: https://github.com/bentoml/OpenLLM/compare/v0.4.24...v0.4.25

Usage

All available models: openllm models

To start a LLM: python -m openllm start HuggingFaceH4/zephyr-7b-beta

To run OpenLLM within a container environment (requires GPUs): docker run --gpus all -it -P -v $PWD/data:$HOME/.cache/huggingface/ ghcr.io/bentoml/openllm:0.4.24 start HuggingFaceH4/zephyr-7b-beta

Find more information about this release in the [CHANGELOG.md](https://github.com/bentoml/OpenLLM/blob/main/CHANGELOG.md)



**Full Changelog**: https://github.com/bentoml/OpenLLM/compare/v0.4.23...v0.4.24

Usage

All available models: openllm models

To start a LLM: python -m openllm start HuggingFaceH4/zephyr-7b-beta

To run OpenLLM within a container environment (requires GPUs): docker run --gpus all -it -P -v $PWD/data:$HOME/.cache/huggingface/ ghcr.io/bentoml/openllm:0.4.23 start HuggingFaceH4/zephyr-7b-beta

Find more information about this release in the [CHANGELOG.md](https://github.com/bentoml/OpenLLM/blob/main/CHANGELOG.md)



What's Changed
* chore: cleanup unused prompt templates by aarnphm in https://github.com/bentoml/OpenLLM/pull/713
* feat(generation): add support for eos_token_id by aarnphm in https://github.com/bentoml/OpenLLM/pull/714
* fix(ci): tests by aarnphm in https://github.com/bentoml/OpenLLM/pull/715
* refactor: delete unused code by aarnphm in https://github.com/bentoml/OpenLLM/pull/716
* chore(logger): fix logger and streamline style by aarnphm in https://github.com/bentoml/OpenLLM/pull/717
* chore(strategy): compact and add stubs by aarnphm in https://github.com/bentoml/OpenLLM/pull/718
* chore(types): append additional types change by aarnphm in https://github.com/bentoml/OpenLLM/pull/719
* fix(base-image): update base image to include cuda for now by aarnphm in https://github.com/bentoml/OpenLLM/pull/720


**Full Changelog**: https://github.com/bentoml/OpenLLM/compare/v0.4.22...v0.4.23