Openllm

Usage

All available models: openllm models

To start a LLM: python -m openllm start opt

To run OpenLLM within a container environment (requires GPUs): docker run --gpus all -it -P ghcr.io/bentoml/openllm:0.3.0 start opt

To run OpenLLM Clojure UI (community-maintained): docker run -p 8420:80 ghcr.io/bentoml/openllm-ui-clojure:0.3.0

Find more information about this release in the [CHANGELOG.md](https://github.com/bentoml/OpenLLM/blob/main/CHANGELOG.md)



What's Changed
* cron(style): run formatter [generated] [skip ci] by aarnphm in https://github.com/bentoml/OpenLLM/pull/257
* feat(vllm): streaming by aarnphm in https://github.com/bentoml/OpenLLM/pull/260
* chore(build): use latest vllm pre-built kernel by aarnphm in https://github.com/bentoml/OpenLLM/pull/261
* chore(deps): bump bentoml/setup-bentoml-action from 59beefe94e2e8f8ebbedf555fc86bd5d1ae0a708 to d4010d8303684b183f45b33c6a44644f81337bdb by dependabot in https://github.com/bentoml/OpenLLM/pull/267
* chore(deps): bump actions/checkout from 3.5.3 to 3.6.0 by dependabot in https://github.com/bentoml/OpenLLM/pull/269
* ci: pre-commit autoupdate [pre-commit.ci] by pre-commit-ci in https://github.com/bentoml/OpenLLM/pull/273
* chore(deps): bump docker/setup-buildx-action from 2.9.1 to 2.10.0 by dependabot in https://github.com/bentoml/OpenLLM/pull/271
* chore(deps): bump aws-actions/configure-aws-credentials from 2.2.0 to 3.0.1 by dependabot in https://github.com/bentoml/OpenLLM/pull/270
* chore(deps): bump taiki-e/install-action from 2.16.0 to 2.17.2 by dependabot in https://github.com/bentoml/OpenLLM/pull/268
* chore(deps): bump mui/icons-material from 5.14.3 to 5.14.6 by dependabot in https://github.com/bentoml/OpenLLM/pull/265
* chore(deps-dev): bump shadow-cljs from 2.25.2 to 2.25.3 by dependabot in https://github.com/bentoml/OpenLLM/pull/262
* chore(deps): bump mui/base from 5.0.0-beta.11 to 5.0.0-beta.12 by dependabot in https://github.com/bentoml/OpenLLM/pull/264
* chore(deps): bump mui/material from 5.14.5 to 5.14.6 by dependabot in https://github.com/bentoml/OpenLLM/pull/266
* chore(deps): bump mui/x-date-pickers from 6.11.2 to 6.12.0 by dependabot in https://github.com/bentoml/OpenLLM/pull/263
* infra: update build dependencies [clojure-ui build] by aarnphm in https://github.com/bentoml/OpenLLM/pull/278
* fix: persistent styling between ruff and yapf by aarnphm in https://github.com/bentoml/OpenLLM/pull/279
* Fixed weird characters over model dropdowns by GutZuFusss in https://github.com/bentoml/OpenLLM/pull/280
* refactor(breaking): unify LLM API by aarnphm in https://github.com/bentoml/OpenLLM/pull/283
* fix(yapf): align weird new lines break [generated] [skip ci] by aarnphm in https://github.com/bentoml/OpenLLM/pull/284
* fix(serving): vllm distributed size by aarnphm in https://github.com/bentoml/OpenLLM/pull/285
* chore(deps): bump sigstore/cosign-installer from 3.1.1 to 3.1.2 by dependabot in https://github.com/bentoml/OpenLLM/pull/296
* chore(deps): bump taiki-e/install-action from 2.17.2 to 2.17.8 by dependabot in https://github.com/bentoml/OpenLLM/pull/295
* chore(deps): bump aquasecurity/trivy-action from 559eb1224e654a86c844a795e6702a0742c60c72 to fbd16365eb88e12433951383f5e99bd901fc618f by dependabot in https://github.com/bentoml/OpenLLM/pull/294
* chore(deps): bump github/codeql-action from 2.21.4 to 2.21.5 by dependabot in https://github.com/bentoml/OpenLLM/pull/292
* chore(deps): bump crazy-max/ghaction-import-gpg from 5.3.0 to 5.4.0 by dependabot in https://github.com/bentoml/OpenLLM/pull/293
* fix(gptq): use upstream integration by aarnphm in https://github.com/bentoml/OpenLLM/pull/297
* ci: pre-commit autoupdate [pre-commit.ci] by pre-commit-ci in https://github.com/bentoml/OpenLLM/pull/298
* chore(deps): bump mui/x-date-pickers from 6.12.0 to 6.12.1 by dependabot in https://github.com/bentoml/OpenLLM/pull/291
* chore(deps): bump mui/icons-material from 5.14.6 to 5.14.7 by dependabot in https://github.com/bentoml/OpenLLM/pull/290
* chore(deps): bump mui/material from 5.14.6 to 5.14.7 by dependabot in https://github.com/bentoml/OpenLLM/pull/289
* chore(deps): bump mui/base from 5.0.0-beta.12 to 5.0.0-beta.13 by dependabot in https://github.com/bentoml/OpenLLM/pull/287
* chore(deps): bump mui/x-data-grid from 6.11.2 to 6.12.1 by dependabot in https://github.com/bentoml/OpenLLM/pull/288


**Full Changelog**: https://github.com/bentoml/OpenLLM/compare/v0.2.27...v0.3.0

Usage

All available models: openllm models

To start a LLM: python -m openllm start opt

To run OpenLLM within a container environment (requires GPUs): docker run --gpus all -it -P ghcr.io/bentoml/openllm:0.2.27 start opt

To run OpenLLM Clojure UI (community-maintained): docker run -p 8420:80 ghcr.io/bentoml/openllm-ui-clojure:0.2.27

Find more information about this release in the [CHANGELOG.md](https://github.com/bentoml/OpenLLM/blob/main/CHANGELOG.md)



What's Changed
* feat: token streaming and SSE support by aarnphm in https://github.com/bentoml/OpenLLM/pull/240
* chore(deps): bump mui/x-data-grid from 6.11.1 to 6.11.2 by dependabot in https://github.com/bentoml/OpenLLM/pull/242
* chore(deps): bump peter-evans/create-pull-request from 4.2.4 to 5.0.2 by dependabot in https://github.com/bentoml/OpenLLM/pull/244
* chore(deps): bump taiki-e/install-action from 2.15.4 to 2.16.0 by dependabot in https://github.com/bentoml/OpenLLM/pull/245
* chore(deps): bump mui/x-date-pickers from 6.0.0 to 6.11.2 by dependabot in https://github.com/bentoml/OpenLLM/pull/243
* refactor: packages by aarnphm in https://github.com/bentoml/OpenLLM/pull/249
* ci: pre-commit autoupdate [pre-commit.ci] by pre-commit-ci in https://github.com/bentoml/OpenLLM/pull/246
* feat(embeddings): Using self-hosted CPU EC2 runner by aarnphm in https://github.com/bentoml/OpenLLM/pull/250
* refactor(contrib): similar namespace [clojure-ui build] by aarnphm in https://github.com/bentoml/OpenLLM/pull/251
* chore: ignore peft and fix adapter loading issue by aarnphm in https://github.com/bentoml/OpenLLM/pull/255


**Full Changelog**: https://github.com/bentoml/OpenLLM/compare/v0.2.26...v0.2.27

Usage

All available models: openllm models

To start a LLM: python -m openllm start opt

To run OpenLLM within a container environment (requires GPUs): docker run --gpus all -it -P ghcr.io/bentoml/openllm:0.2.26 start opt

To run OpenLLM Clojure UI (community-maintained): docker run -p 8420:80 ghcr.io/bentoml/openllm-ui-clojure:0.2.26

Find more information about this release in the [CHANGELOG.md](https://github.com/bentoml/OpenLLM/blob/main/CHANGELOG.md)



What's Changed
* chore(gh): use setup-bentoml-action by aarnphm in https://github.com/bentoml/OpenLLM/pull/230
* fix(binary): move to correct folders when building standalone installer by aarnphm in https://github.com/bentoml/OpenLLM/pull/228
* chore: conditional commit for running jobs by aarnphm in https://github.com/bentoml/OpenLLM/pull/232
* feat(embedding): Adding generic endpoint by aarnphm in https://github.com/bentoml/OpenLLM/pull/227


**Full Changelog**: https://github.com/bentoml/OpenLLM/compare/v0.2.25...v0.2.26

Usage

All available models: openllm models

To start a LLM: python -m openllm start opt

To run OpenLLM within a container environment (requires GPUs): docker run --gpus all -it -P ghcr.io/bentoml/openllm:0.2.25 start opt

To run OpenLLM Clojure UI (community-maintained): docker run -p 8420:80 ghcr.io/bentoml/openllm-ui-clojure:0.2.25

Find more information about this release in the [CHANGELOG.md](https://github.com/bentoml/OpenLLM/blob/main/CHANGELOG.md)

What's Changed
* chore: upload nightly wheels to test.pypi.org by aarnphm in https://github.com/bentoml/OpenLLM/pull/215
* feat(contrib): ClojureScript UI by GutZuFusss in https://github.com/bentoml/OpenLLM/pull/89
* fix(ci): remove broken build hooks by aarnphm in https://github.com/bentoml/OpenLLM/pull/216
* chore(ci): add dependabot and fix vllm release container by aarnphm in https://github.com/bentoml/OpenLLM/pull/217
* feat(models): add vLLM support for Falcon by aarnphm in https://github.com/bentoml/OpenLLM/pull/223
* chore(readme): update nightly badge [skip ci] by aarnphm in https://github.com/bentoml/OpenLLM/pull/224

New Contributors
* GutZuFusss made their first contribution in https://github.com/bentoml/OpenLLM/pull/89

**Full Changelog**: https://github.com/bentoml/OpenLLM/compare/v0.2.24...v0.2.25

Usage

All available models: openllm models

To start a LLM: python -m openllm start opt

To run OpenLLM within a container environment (requires GPUs): docker run --gpus all -it --entrypoint=/bin/bash -P ghcr.io/bentoml/openllm:0.2.22 openllm --help

Find more information about this release in the [CHANGELOG.md](https://github.com/bentoml/OpenLLM/blob/main/CHANGELOG.md)



**Full Changelog**: https://github.com/bentoml/OpenLLM/compare/v0.2.21...v0.2.22

Usage

All available models: openllm models

To start a LLM: python -m openllm start opt

To run OpenLLM within a container environment (requires GPUs): docker run --gpus all -it --entrypoint=/bin/bash -P ghcr.io/bentoml/openllm:0.2.21 openllm --help

Find more information about this release in the [CHANGELOG.md](https://github.com/bentoml/OpenLLM/blob/main/CHANGELOG.md)



What's Changed
* fix(release): fix exclude options within compiled wheels by aarnphm in https://github.com/bentoml/OpenLLM/pull/197
* infra: migrate to initial `openllm-node` library by aarnphm in https://github.com/bentoml/OpenLLM/pull/199
* perf: compiled modules and enable lazyeval by aarnphm in https://github.com/bentoml/OpenLLM/pull/200


**Full Changelog**: https://github.com/bentoml/OpenLLM/compare/v0.2.20...v0.2.21