What's Changed
* Modify completion handler for SageMaker to use payload from `prepared_request` by andjsmi in https://github.com/BerriAI/litellm/pull/9326
* Arize integration Fix by nate-mar in https://github.com/BerriAI/litellm/pull/9338
* Fix get_llm_provider error + add model name to tpm/rpm cache key (enables wildcard models to work w/ usage-based-routing) by krrishdholakia in https://github.com/BerriAI/litellm/pull/9355
* Litellm dev 03 18 2025 p1 v3 by krrishdholakia in https://github.com/BerriAI/litellm/pull/9354
* [Bug Fix] Arize AI Logging Integration with LiteLLM by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9352
* build(model_prices_and_context_window.json): fix azure gpt-4o pricing by krrishdholakia in https://github.com/BerriAI/litellm/pull/9361
* Contributor PR: Fix sagemaker too little data for content error by krrishdholakia in https://github.com/BerriAI/litellm/pull/9335
* [Feat] - API - Allow using dynamic Arize AI Spaces on LiteLLM by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9353
* fix(common_utils.py): handle cris only model by krrishdholakia in https://github.com/BerriAI/litellm/pull/9363
* docs(litellm_proxy): correct parameter assignment in litellm proxy docs by colesmcintosh in https://github.com/BerriAI/litellm/pull/9375
* Feature flag checking LiteLLM_CredentialsTable by krrishdholakia in https://github.com/BerriAI/litellm/pull/9376
* fix(lowest_tpm_rpm_v2.py): support batch writing increments to redis by krrishdholakia in https://github.com/BerriAI/litellm/pull/9357
* Support 'prisma migrate' for db schema changes by krrishdholakia in https://github.com/BerriAI/litellm/pull/9382
* Fix latency redis by emerzon in https://github.com/BerriAI/litellm/pull/9387
* Revert "Fix latency redis" by krrishdholakia in https://github.com/BerriAI/litellm/pull/9388
* build(model_prices_and_context_window.json): add o1-pro pricing by krrishdholakia in https://github.com/BerriAI/litellm/pull/9397
* [Bug Fix] - Azure OpenAI - ensure SSL verification runs by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9341
* [Feat] - Allow building custom prompt management integration by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9384
* Litellm fix icons by azdolinski in https://github.com/BerriAI/litellm/pull/9374
* [UI Improvement] Use local icons for model providers instead of downloading them by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9399
* fix(internal_user_endpoints.py): re-introduce upsert on user not found by krrishdholakia in https://github.com/BerriAI/litellm/pull/9395
* docs: Include Phoenix Page into sidebar under observability by SrilakshmiC in https://github.com/BerriAI/litellm/pull/9332
* fix(types/utils.py): support openai 'file' message type by krrishdholakia in https://github.com/BerriAI/litellm/pull/9402
* feat: Add support for custom OPENROUTER_API_BASE via get_secret in co… by graysonchen in https://github.com/BerriAI/litellm/pull/9369
* fix: VertexAI outputDimensionality configuration by JamesGuthrie in https://github.com/BerriAI/litellm/pull/9366
* docs(vertexai): fix typo in required env variables by Shiti in https://github.com/BerriAI/litellm/pull/9325
* Update perplexity.md by jollyolliel in https://github.com/BerriAI/litellm/pull/9290
* fix: VoyageAI `prompt_token` always empty by lucasra1 in https://github.com/BerriAI/litellm/pull/9260
* build(deps): bump litellm from 1.55.3 to 1.61.15 in /cookbook/litellm-ollama-docker-image by dependabot in https://github.com/BerriAI/litellm/pull/9422
* [Feat] OpenAI o1-pro Responses API streaming support by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9419
* [Feat] Add OpenAI o1-pro support on Responses API by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9415
* [Docs - Draft] LiteLLM x MCP Interface by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9435
* support returning api-base on pass-through endpoints + consistently return 404 if team not found in DB by krrishdholakia in https://github.com/BerriAI/litellm/pull/9439
* fix(handle_error.py): make cooldown error more descriptive by krrishdholakia in https://github.com/BerriAI/litellm/pull/9438
* Consistent anthropic response_format streaming/non-streaming behaviour by krrishdholakia in https://github.com/BerriAI/litellm/pull/9437
* New Azure Models (GPT-4.5-Preview, Mistral Small 3.1) by emerzon in https://github.com/BerriAI/litellm/pull/9453
* Set max size limit to in-memory cache item - prevents OOM errors by krrishdholakia in https://github.com/BerriAI/litellm/pull/9448
* fix(model_param_helper.py): update `_get_litellm_supported_transcription_kwargs()` to use proper annotations from `TranscriptionCreateParamsNonStreaming` & ``TranscriptionCreateParamsStreaming` by hsaeed3 in https://github.com/BerriAI/litellm/pull/9451
* [Feat] LiteLLM x MCP Bridge - Use MCP Tools with LiteLLM by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9436
* fix(model_param_helper.py): update _get_litellm_supported_transcription_kwargs() to use proper annotations from TranscriptionCreateParamsNonStreaming & `TranscriptionCreateParamsStreaming by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9455
New Contributors
* andjsmi made their first contribution in https://github.com/BerriAI/litellm/pull/9326
* azdolinski made their first contribution in https://github.com/BerriAI/litellm/pull/9374
* SrilakshmiC made their first contribution in https://github.com/BerriAI/litellm/pull/9332
* graysonchen made their first contribution in https://github.com/BerriAI/litellm/pull/9369
* JamesGuthrie made their first contribution in https://github.com/BerriAI/litellm/pull/9366
* Shiti made their first contribution in https://github.com/BerriAI/litellm/pull/9325
* jollyolliel made their first contribution in https://github.com/BerriAI/litellm/pull/9290
* hsaeed3 made their first contribution in https://github.com/BerriAI/litellm/pull/9451
**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.63.12-nightly...v1.63.14.rc
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.63.14.rc
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 230.0 | 249.89683940656786 | 6.2068378536570386 | 0.0 | 1857 | 0 | 196.31636200006142 | 3500.2345190000597 |
| Aggregated | Passed ✅ | 230.0 | 249.89683940656786 | 6.2068378536570386 | 0.0 | 1857 | 0 | 196.31636200006142 | 3500.2345190000597 |
v1.63.14-nightly
What's Changed
* Modify completion handler for SageMaker to use payload from `prepared_request` by andjsmi in https://github.com/BerriAI/litellm/pull/9326
* Arize integration Fix by nate-mar in https://github.com/BerriAI/litellm/pull/9338
* Fix get_llm_provider error + add model name to tpm/rpm cache key (enables wildcard models to work w/ usage-based-routing) by krrishdholakia in https://github.com/BerriAI/litellm/pull/9355
* Litellm dev 03 18 2025 p1 v3 by krrishdholakia in https://github.com/BerriAI/litellm/pull/9354
* [Bug Fix] Arize AI Logging Integration with LiteLLM by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9352
* build(model_prices_and_context_window.json): fix azure gpt-4o pricing by krrishdholakia in https://github.com/BerriAI/litellm/pull/9361
* Contributor PR: Fix sagemaker too little data for content error by krrishdholakia in https://github.com/BerriAI/litellm/pull/9335
* [Feat] - API - Allow using dynamic Arize AI Spaces on LiteLLM by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9353
* fix(common_utils.py): handle cris only model by krrishdholakia in https://github.com/BerriAI/litellm/pull/9363
* docs(litellm_proxy): correct parameter assignment in litellm proxy docs by colesmcintosh in https://github.com/BerriAI/litellm/pull/9375
* Feature flag checking LiteLLM_CredentialsTable by krrishdholakia in https://github.com/BerriAI/litellm/pull/9376
* fix(lowest_tpm_rpm_v2.py): support batch writing increments to redis by krrishdholakia in https://github.com/BerriAI/litellm/pull/9357
* Support 'prisma migrate' for db schema changes by krrishdholakia in https://github.com/BerriAI/litellm/pull/9382
* Fix latency redis by emerzon in https://github.com/BerriAI/litellm/pull/9387
* Revert "Fix latency redis" by krrishdholakia in https://github.com/BerriAI/litellm/pull/9388
* build(model_prices_and_context_window.json): add o1-pro pricing by krrishdholakia in https://github.com/BerriAI/litellm/pull/9397
* [Bug Fix] - Azure OpenAI - ensure SSL verification runs by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9341
* [Feat] - Allow building custom prompt management integration by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9384
* Litellm fix icons by azdolinski in https://github.com/BerriAI/litellm/pull/9374
* [UI Improvement] Use local icons for model providers instead of downloading them by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9399
* fix(internal_user_endpoints.py): re-introduce upsert on user not found by krrishdholakia in https://github.com/BerriAI/litellm/pull/9395
* docs: Include Phoenix Page into sidebar under observability by SrilakshmiC in https://github.com/BerriAI/litellm/pull/9332
* fix(types/utils.py): support openai 'file' message type by krrishdholakia in https://github.com/BerriAI/litellm/pull/9402
* feat: Add support for custom OPENROUTER_API_BASE via get_secret in co… by graysonchen in https://github.com/BerriAI/litellm/pull/9369
* fix: VertexAI outputDimensionality configuration by JamesGuthrie in https://github.com/BerriAI/litellm/pull/9366
* docs(vertexai): fix typo in required env variables by Shiti in https://github.com/BerriAI/litellm/pull/9325
* Update perplexity.md by jollyolliel in https://github.com/BerriAI/litellm/pull/9290
* fix: VoyageAI `prompt_token` always empty by lucasra1 in https://github.com/BerriAI/litellm/pull/9260
* build(deps): bump litellm from 1.55.3 to 1.61.15 in /cookbook/litellm-ollama-docker-image by dependabot in https://github.com/BerriAI/litellm/pull/9422
* [Feat] OpenAI o1-pro Responses API streaming support by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9419
* [Feat] Add OpenAI o1-pro support on Responses API by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9415
* [Docs - Draft] LiteLLM x MCP Interface by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9435
* support returning api-base on pass-through endpoints + consistently return 404 if team not found in DB by krrishdholakia in https://github.com/BerriAI/litellm/pull/9439
* fix(handle_error.py): make cooldown error more descriptive by krrishdholakia in https://github.com/BerriAI/litellm/pull/9438
* Consistent anthropic response_format streaming/non-streaming behaviour by krrishdholakia in https://github.com/BerriAI/litellm/pull/9437
* New Azure Models (GPT-4.5-Preview, Mistral Small 3.1) by emerzon in https://github.com/BerriAI/litellm/pull/9453
* Set max size limit to in-memory cache item - prevents OOM errors by krrishdholakia in https://github.com/BerriAI/litellm/pull/9448
* fix(model_param_helper.py): update `_get_litellm_supported_transcription_kwargs()` to use proper annotations from `TranscriptionCreateParamsNonStreaming` & ``TranscriptionCreateParamsStreaming` by hsaeed3 in https://github.com/BerriAI/litellm/pull/9451
* [Feat] LiteLLM x MCP Bridge - Use MCP Tools with LiteLLM by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9436
* fix(model_param_helper.py): update _get_litellm_supported_transcription_kwargs() to use proper annotations from TranscriptionCreateParamsNonStreaming & `TranscriptionCreateParamsStreaming by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9455
New Contributors
* andjsmi made their first contribution in https://github.com/BerriAI/litellm/pull/9326
* azdolinski made their first contribution in https://github.com/BerriAI/litellm/pull/9374
* SrilakshmiC made their first contribution in https://github.com/BerriAI/litellm/pull/9332
* graysonchen made their first contribution in https://github.com/BerriAI/litellm/pull/9369
* JamesGuthrie made their first contribution in https://github.com/BerriAI/litellm/pull/9366
* Shiti made their first contribution in https://github.com/BerriAI/litellm/pull/9325
* jollyolliel made their first contribution in https://github.com/BerriAI/litellm/pull/9290
* hsaeed3 made their first contribution in https://github.com/BerriAI/litellm/pull/9451
**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.63.12-nightly...v1.63.14-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.63.14-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 210.0 | 225.80703369680768 | 6.282456357875219 | 0.0 | 1880 | 0 | 182.8801230000181 | 2803.2175499999994 |
| Aggregated | Passed ✅ | 210.0 | 225.80703369680768 | 6.282456357875219 | 0.0 | 1880 | 0 | 182.8801230000181 | 2803.2175499999994 |
v1.63.12-nightly
What's Changed
* Fixes bedrock modelId encoding for Inference Profiles by omrishiv in https://github.com/BerriAI/litellm/pull/9123
* Aim Security post-call guardrails support by hxtomer in https://github.com/BerriAI/litellm/pull/8356
* Litellm dev 03 12 2025 contributor prs p2 by krrishdholakia in https://github.com/BerriAI/litellm/pull/9216
* Support bedrock Application inference profiles + Support guardrails on streaming responses by krrishdholakia in https://github.com/BerriAI/litellm/pull/9274
* v1.63.11-stable release notes by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9279
* Infer aws region from bedrock application profile id by krrishdholakia in https://github.com/BerriAI/litellm/pull/9281
* feat: make masterkey secret configurable by mknet3 in https://github.com/BerriAI/litellm/pull/9288
* fix(utils.py): Prevents final chunk w/ usage from being ignored by krrishdholakia in https://github.com/BerriAI/litellm/pull/9314
* Update prompt_caching.md to fix typo by afspies in https://github.com/BerriAI/litellm/pull/9317
* fix(redis_cache.py): add 5s default timeout by krrishdholakia in https://github.com/BerriAI/litellm/pull/9322
* Support reading litellm proxy response cost header in sdk + support setting lower ssl security level by krrishdholakia in https://github.com/BerriAI/litellm/pull/9330
* [Bug fix] Reset Budget Job by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9329
* fix(ollama/completions/transformation.py): pass prompt, untemplated o… by krrishdholakia in https://github.com/BerriAI/litellm/pull/9333
* [UI] - Allow controlling default internal user settings on ui by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9328
* [Patch] - Allow disabling all spend updates / writes to DB by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9331
New Contributors
* afspies made their first contribution in https://github.com/BerriAI/litellm/pull/9317
**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.63.11-nightly...v1.63.12-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.63.12-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 240.0 | 266.99215766025395 | 6.081096542088128 | 0.0 | 1819 | 0 | 211.12568599994574 | 4206.960361000029 |
| Aggregated | Passed ✅ | 240.0 | 266.99215766025395 | 6.081096542088128 | 0.0 | 1819 | 0 | 211.12568599994574 | 4206.960361000029 |
v1.63.11-stable.patch1
**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.63.11-nightly...v1.63.11-stable.patch1
🚨 Known issue on Azure OpenAI - We don't recommend upgrading if you use Azure OpenAI. This version failed our Azure OpenAI load test
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.63.11-stable.patch1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 250.0 | 286.0849351151513 | 6.065608931564817 | 0.0 | 1815 | 0 | 215.73641000003363 | 4467.043058000001 |
| Aggregated | Passed ✅ | 250.0 | 286.0849351151513 | 6.065608931564817 | 0.0 | 1815 | 0 | 215.73641000003363 | 4467.043058000001 |
v1.63.11-stable
Docker Run LiteLLM Proxy
🚨 Known issue on Azure OpenAI - We don't recommend upgrading if you use Azure OpenAI. This version failed our Azure OpenAI load test
docker run
-e STORE_MODEL_IN_DB=True
-p 4000:4000
ghcr.io/berriai/litellm:main-v1.63.11-stable
These are the changes since v1.63.2-stable.
This release is primarily focused on:
- [Beta] Responses API Support
- Snowflake Cortex Support, Amazon Nova Image Generation
- UI - Credential Management, re-use credentials when adding new models
- UI - Test Connection to LLM Provider before adding a model
Demo Instance
Here's a Demo Instance to test changes:
- Instance: https://demo.litellm.ai/
- Login Credentials:
- Username: admin
- Password: sk-1234
New Models / Updated Models
- Image Generation support for Amazon Nova Canvas [Getting Started](https://docs.litellm.ai/docs/providers/bedrock#image-generation)
- Add pricing for Jamba new models [PR](https://github.com/BerriAI/litellm/pull/9032/files)
- Add pricing for Amazon EU models [PR](https://github.com/BerriAI/litellm/pull/9056/files)
- Add Bedrock Deepseek R1 model pricing [PR](https://github.com/BerriAI/litellm/pull/9108/files)
- Update Gemini pricing: Gemma 3, Flash 2 thinking update, LearnLM [PR](https://github.com/BerriAI/litellm/pull/9190/files)
- Mark Cohere Embedding 3 models as Multimodal [PR](https://github.com/BerriAI/litellm/pull/9176/commits/c9a576ce4221fc6e50dc47cdf64ab62736c9da41)
- Add Azure Data Zone pricing [PR](https://github.com/BerriAI/litellm/pull/9185/files#diff-19ad91c53996e178c1921cbacadf6f3bae20cfe062bd03ee6bfffb72f847ee37)
- LiteLLM Tracks cost for `azure/eu` and `azure/us` models
LLM Translation
1. **New Endpoints**
- [Beta] POST `/responses` API. [Getting Started](https://docs.litellm.ai/docs/response_api)
2. **New LLM Providers**
- Snowflake Cortex [Getting Started](https://docs.litellm.ai/docs/providers/snowflake)
3. **New LLM Features**
- Support OpenRouter `reasoning_content` on streaming [Getting Started](https://docs.litellm.ai/docs/reasoning_content)
4. **Bug Fixes**
- OpenAI: Return `code`, `param` and `type` on bad request error [More information on litellm exceptions](https://docs.litellm.ai/docs/exception_mapping)
- Bedrock: Fix converse chunk parsing to only return empty dict on tool use [PR](https://github.com/BerriAI/litellm/pull/9166)
- Bedrock: Support extra_headers [PR](https://github.com/BerriAI/litellm/pull/9113)
- Azure: Fix Function Calling Bug & Update Default API Version to `2025-02-01-preview` [PR](https://github.com/BerriAI/litellm/pull/9191)
- Azure: Fix AI services URL [PR](https://github.com/BerriAI/litellm/pull/9185)
- Vertex AI: Handle HTTP 201 status code in response [PR](https://github.com/BerriAI/litellm/pull/9193)
- Perplexity: Fix incorrect streaming response [PR](https://github.com/BerriAI/litellm/pull/9081)
- Triton: Fix streaming completions bug [PR](https://github.com/BerriAI/litellm/pull/8386)
- Deepgram: Support bytes.IO when handling audio files for transcription [PR](https://github.com/BerriAI/litellm/pull/9071)
- Ollama: Fix "system" role has become unacceptable [PR](https://github.com/BerriAI/litellm/pull/9261)
- All Providers (Streaming): Fix String `data:` stripped from entire content in streamed responses [PR](https://github.com/BerriAI/litellm/pull/9070)
Spend Tracking Improvements
1. Support Bedrock converse cache token tracking [Getting Started](https://docs.litellm.ai/docs/completion/prompt_caching)
2. Cost Tracking for Responses API [Getting Started](https://docs.litellm.ai/docs/response_api)
3. Fix Azure Whisper cost tracking [Getting Started](https://docs.litellm.ai/docs/audio_transcription)
UI
Re-Use Credentials on UI
You can now onboard LLM provider credentials on LiteLLM UI. Once these credentials are added you can re-use them when adding new models [Getting Started](https://docs.litellm.ai/docs/proxy/ui_credentials)
Test Connections before adding models
Before adding a model you can test the connection to the LLM provider to verify you have setup your API Base + API Key correctly
General UI Improvements
1. Add Models Page
- Allow adding Cerebras, Sambanova, Perplexity, Fireworks, Openrouter, TogetherAI Models, Text-Completion OpenAI on Admin UI
- Allow adding EU OpenAI models
- Fix: Instantly show edit + deletes to models
2. Keys Page
- Fix: Instantly show newly created keys on Admin UI (don't require refresh)
- Fix: Allow clicking into Top Keys when showing users Top API Key
- Fix: Allow Filter Keys by Team Alias, Key Alias and Org
- UI Improvements: Show 100 Keys Per Page, Use full height, increase width of key alias
3. Users Page
- Fix: Show correct count of internal user keys on Users Page
- Fix: Metadata not updating in Team UI
4. Logs Page
- UI Improvements: Keep expanded log in focus on LiteLLM UI
- UI Improvements: Minor improvements to logs page
- Fix: Allow internal user to query their own logs
- Allow switching off storing Error Logs in DB [Getting Started](https://docs.litellm.ai/docs/proxy/ui_logs)
5. Sign In/Sign Out
- Fix: Correctly use `PROXY_LOGOUT_URL` when set [Getting Started](https://docs.litellm.ai/docs/proxy/self_serve#setting-custom-logout-urls)
Security
1. Support for Rotating Master Keys [Getting Started](https://docs.litellm.ai/docs/proxy/master_key_rotations)
2. Fix: Internal User Viewer Permissions, don't allow `internal_user_viewer` role to see `Test Key Page` or `Create Key Button` [More information on role based access controls](https://docs.litellm.ai/docs/proxy/access_control)
3. Emit audit logs on All user + model Create/Update/Delete endpoints [Getting Started](https://docs.litellm.ai/docs/proxy/multiple_admins)
4. JWT
- Support multiple JWT OIDC providers [Getting Started](https://docs.litellm.ai/docs/proxy/token_auth)
- Fix JWT access with Groups not working when team is assigned All Proxy Models access
5. Using K/V pairs in 1 AWS Secret [Getting Started](https://docs.litellm.ai/docs/secret#using-kv-pairs-in-1-aws-secret)
Logging Integrations
1. Prometheus: Track Azure LLM API latency metric [Getting Started](https://docs.litellm.ai/docs/proxy/prometheus#request-latency-metrics)
2. Athina: Added tags, user_feedback and model_options to additional_keys which can be sent to Athina [Getting Started](https://docs.litellm.ai/docs/observability/athina_integration)
Performance / Reliability improvements
1. Redis + litellm router - Fix Redis cluster mode for litellm router [PR](https://github.com/BerriAI/litellm/pull/9010)
General Improvements
1. OpenWebUI Integration - display `thinking` tokens
- Guide on getting started with LiteLLM x OpenWebUI. [Getting Started](https://docs.litellm.ai/docs/tutorials/openweb_ui)
- Display `thinking` tokens on OpenWebUI (Bedrock, Anthropic, Deepseek) [Getting Started](https://docs.litellm.ai/docs/tutorials/openweb_ui#render-thinking-content-on-openweb-ui)
<Image img={require('../../img/litellm_thinking_openweb.gif')} />
New Contributors
* ogunoz made their first contribution in https://github.com/BerriAI/litellm/pull/9010
* Askir made their first contribution in https://github.com/BerriAI/litellm/pull/8576
* tvishwanadha made their first contribution in https://github.com/BerriAI/litellm/pull/9071
* 5aaee9 made their first contribution in https://github.com/BerriAI/litellm/pull/9081
* mounta11n made their first contribution in https://github.com/BerriAI/litellm/pull/8757
* minwhoo made their first contribution in https://github.com/BerriAI/litellm/pull/8386
* santibreo made their first contribution in https://github.com/BerriAI/litellm/pull/7736
* utkashd made their first contribution in https://github.com/BerriAI/litellm/pull/8956
* kearnsw made their first contribution in https://github.com/BerriAI/litellm/pull/9108
* sfarthin made their first contribution in https://github.com/BerriAI/litellm/pull/8019
* lucasra1 made their first contribution in https://github.com/BerriAI/litellm/pull/9180
* youngchannelforyou made their first contribution in https://github.com/BerriAI/litellm/pull/9193
* xucailiang made their first contribution in https://github.com/BerriAI/litellm/pull/8741
* SunnyWan59 made their first contribution in https://github.com/BerriAI/litellm/pull/8950
* bexelbie made their first contribution in https://github.com/BerriAI/litellm/pull/9254
* briandevvn made their first contribution in https://github.com/BerriAI/litellm/pull/9261
**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.63.2-stable...v1.63.11-stable
Github generated changelog
* Fix redis cluster mode for routers by ogunoz in https://github.com/BerriAI/litellm/pull/9010
* [Feat] - Display `thinking` tokens on OpenWebUI (Bedrock, Anthropic, Deepseek) by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9029
* (AWS Secret Manager) - Using K/V pairs in 1 AWS Secret by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9039
* (Docs) connect litellm to open web ui by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9040
* Added PDL project by vazirim in https://github.com/BerriAI/litellm/pull/8925
* (UI) - Allow adding EU OpenAI models by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9042
* fix(team_endpoints.py): ensure 404 raised when team not found + fix setting tags on keys by krrishdholakia in https://github.com/BerriAI/litellm/pull/9038
* build(model_prices_and_context_window.json): update azure o1 mini pri… by krrishdholakia in https://github.com/BerriAI/litellm/pull/9046
* Support master key rotations by krrishdholakia in https://github.com/BerriAI/litellm/pull/9041
* (Feat) - add pricing for eu.amazon.nova models by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9056
* docs: Add project page for pgai by Askir in https://github.com/BerriAI/litellm/pull/8576
* Mark several Claude models as being able to accept PDF inputs by minhduc0711 in https://github.com/BerriAI/litellm/pull/9054
* (UI) - Keys Page - Show 100 Keys Per Page, Use full height, increase width of key alias by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9064
* (UI) Logs Page - Keep expanded log in focus on LiteLLM UI by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9061
* (Docs) OpenWeb x LiteLLM Docker compose + Instructions on spend tracking + logging by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9059
* (UI) - Allow adding Cerebras, Sambanova, Perplexity, Fireworks, Openrouter, TogetherAI Models on Admin UI by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9069
* UI - new API Playground for testing LiteLLM translation by krrishdholakia in https://github.com/BerriAI/litellm/pull/9073
* Bug fix - String data: stripped from entire content in streamed Gemini responses by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9070
* (UI) - Minor improvements to logs page by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9076
* Bug fix: support bytes.IO when handling audio files for transcription by tvishwanadha in https://github.com/BerriAI/litellm/pull/9071
* Fix batches api cost tracking + Log batch models in spend logs / standard logging payload by krrishdholakia in https://github.com/BerriAI/litellm/pull/9077
* (UI) - Fix, Allow Filter Keys by Team Alias, Key Alias and Org by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9083
* (Clean up) - Allow switching off storing Error Logs in DB by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9084
* (UI) - Fix show correct count of internal user keys on Users Page by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9082
* New stable release notes by krrishdholakia in https://github.com/BerriAI/litellm/pull/9085
* Litellm dev 03 08 2025 p3 by krrishdholakia in https://github.com/BerriAI/litellm/pull/9089
* feat: prioritize api_key over tenant_id for more Azure AD token provi… by krrishdholakia in https://github.com/BerriAI/litellm/pull/8701
* Fix incorrect streaming response by 5aaee9 in https://github.com/BerriAI/litellm/pull/9081
* Support openrouter `reasoning_content` on streaming by krrishdholakia in https://github.com/BerriAI/litellm/pull/9094
* add support for Amazon Nova Canvas model by omrishiv in https://github.com/BerriAI/litellm/pull/7838
* pricing for jamba new models by themrzmaster in https://github.com/BerriAI/litellm/pull/9032
* build(deps): bump jinja2 from 3.1.4 to 3.1.6 by dependabot in https://github.com/BerriAI/litellm/pull/9014
* (docs) add section for contributing to litellm by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9107
* build: Add Makefile for LiteLLM project with test targets by colesmcintosh in https://github.com/BerriAI/litellm/pull/8948
* (Docs) - Contributing to litellm by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9110
* Added tags, user_feedback and model_options to additional_keys which can be sent to athina by vivek-athina in https://github.com/BerriAI/litellm/pull/8845
* fix missing comma by niinpatel in https://github.com/BerriAI/litellm/pull/8746
* Update model_prices_and_context_window.json by mounta11n in https://github.com/BerriAI/litellm/pull/8757
* Fix triton streaming completions bug by minwhoo in https://github.com/BerriAI/litellm/pull/8386
* (docs) Update vertex.md old code example by santibreo in https://github.com/BerriAI/litellm/pull/7736
* (Feat) - Allow adding Text-Completion OpenAI models through UI by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9102
* docs(pr-template): update unit test command in checklist by colesmcintosh in https://github.com/BerriAI/litellm/pull/9119
* [UI SSO Bug fix] - Correctly use `PROXY_LOGOUT_URL` when set by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9117
* Validate `model_prices_and_context_window.json` with a test, clarify possible `mode` values + ensure consistent use of `mode` by utkashd in https://github.com/BerriAI/litellm/pull/8956
* JWT Auth Fix - [Bug]: JWT access with Groups not working when team is assigned All Proxy Models access by ishaan-jaff in https://github.com/BerriAI/litellm/pull/8934
* fix(base_invoke_transformation.py): support extra_headers on bedrock … by krrishdholakia in https://github.com/BerriAI/litellm/pull/9113
* feat(handle_jwt.py): support multiple jwt url's by krrishdholakia in https://github.com/BerriAI/litellm/pull/9047
* Return `code`, `param` and `type` on openai bad request error by krrishdholakia in https://github.com/BerriAI/litellm/pull/9109
* feature: Handle ManagedIdentityCredential in Azure AD token provider by you-n-g in https://github.com/BerriAI/litellm/pull/9135
* Adding/Update of models by emerzon in https://github.com/BerriAI/litellm/pull/9120
* Update bedrock.md for variable consistency by superpoussin22 in https://github.com/BerriAI/litellm/pull/8185
* ci: add helm unittest by mknet3 in https://github.com/BerriAI/litellm/pull/9068
* [UI Fixes RBAC] - for Internal User Viewer Permissions by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9148
* Delegate router azure client init logic to azure provider by krrishdholakia in https://github.com/BerriAI/litellm/pull/9140
* feat: add bedrock deepseek r1 model pricing by kearnsw in https://github.com/BerriAI/litellm/pull/9108
* fix(internal_user_endpoints.py): allow internal user to query their o… by krrishdholakia in https://github.com/BerriAI/litellm/pull/9162
* add support for Amazon Nova Canvas model (7838) by krrishdholakia in https://github.com/BerriAI/litellm/pull/9101
* Fix bedrock chunk parsing + azure whisper cost tracking by krrishdholakia in https://github.com/BerriAI/litellm/pull/9166
* Bing Search Pass Thru by sfarthin in https://github.com/BerriAI/litellm/pull/8019
* [Feat] Add OpenAI Responses API to litellm python SDK by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9155
* Support credential management on Proxy - via CRUD endpoints - `credentials/*` by krrishdholakia in https://github.com/BerriAI/litellm/pull/9124
* Bump babel/runtime-corejs3 from 7.26.0 to 7.26.10 in /docs/my-website by dependabot in https://github.com/BerriAI/litellm/pull/9167
* Bump babel/helpers from 7.26.0 to 7.26.10 in /docs/my-website by dependabot in https://github.com/BerriAI/litellm/pull/9168
* fix(azure): Patch for Function Calling Bug & Update Default API Version to `2025-02-01-preview` by colesmcintosh in https://github.com/BerriAI/litellm/pull/9191
* [Feat] - Add Responses API on LiteLLM Proxy by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9183
* gemini price updates: gemma 3, flash 2 thinking update, learnlm by yigitkonur in https://github.com/BerriAI/litellm/pull/9190
* Mark Cohere Embedding 3 models as Multimodal by emerzon in https://github.com/BerriAI/litellm/pull/9176
* Fix Metadata not updating in Team UI by lucasra1 in https://github.com/BerriAI/litellm/pull/9180
* feat: initial commit adding support for credentials on proxy ui by krrishdholakia in https://github.com/BerriAI/litellm/pull/9186
* Fix azure ai services url + add azure data zone pricing by krrishdholakia in https://github.com/BerriAI/litellm/pull/9185
* (gemini)Handle HTTP 201 status code in Vertex AI response by youngchannelforyou in https://github.com/BerriAI/litellm/pull/9193
* feat/postgres-volumes by xucailiang in https://github.com/BerriAI/litellm/pull/8741
* [FEAT] Support for Snowflake REST API LLMs 7979 by SunnyWan59 in https://github.com/BerriAI/litellm/pull/8950
* fix(azure.py): track azure llm api latency metric by krrishdholakia in https://github.com/BerriAI/litellm/pull/9217
* Support bedrock converse cache token tracking by krrishdholakia in https://github.com/BerriAI/litellm/pull/9221
* Emit audit logs on All user + model Create/Update/Delete endpoints by krrishdholakia in https://github.com/BerriAI/litellm/pull/9223
* (UI Usage) - Allow clicking into Top Keys when showing users Top API Key by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9225
* [Feat] Add Snowflake Cortex to LiteLLM by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9222
* [Fixes] Responses API - allow /responses and subpaths as LLM API route + Add exception mapping for responses API by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9220
* docs: Add centralized credential management docs by bexelbie in https://github.com/BerriAI/litellm/pull/9254
* Docs: Update configs.md by bexelbie in https://github.com/BerriAI/litellm/pull/9263
* Support reusing existing model credentials by krrishdholakia in https://github.com/BerriAI/litellm/pull/9267
* LiteLLM UI Fixes by krrishdholakia in https://github.com/BerriAI/litellm/pull/9269
* Fix "system" role has become unacceptable in ollama by briandevvn in https://github.com/BerriAI/litellm/pull/9261
* Litellm rc 03 14 2025 patch 1 by krrishdholakia in https://github.com/BerriAI/litellm/pull/9271
* [Feat] UI - Add Test Connection by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9272
* [UI] Fix 1 - instantly show newly create keys on Admin UI (don't require refresh) by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9257
* (UI) Fix model edit + delete - instantly show edit + deletes to models by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9258
v1.63.11-nightly
What's Changed
* gemini price updates: gemma 3, flash 2 thinking update, learnlm by yigitkonur in https://github.com/BerriAI/litellm/pull/9190
* Mark Cohere Embedding 3 models as Multimodal by emerzon in https://github.com/BerriAI/litellm/pull/9176
* Fix Metadata not updating in Team UI by lucasra1 in https://github.com/BerriAI/litellm/pull/9180
* feat: initial commit adding support for credentials on proxy ui by krrishdholakia in https://github.com/BerriAI/litellm/pull/9186
* Fix azure ai services url + add azure data zone pricing by krrishdholakia in https://github.com/BerriAI/litellm/pull/9185
* (gemini)Handle HTTP 201 status code in Vertex AI response by youngchannelforyou in https://github.com/BerriAI/litellm/pull/9193
* feat/postgres-volumes by xucailiang in https://github.com/BerriAI/litellm/pull/8741
* [FEAT] Support for Snowflake REST API LLMs 7979 by SunnyWan59 in https://github.com/BerriAI/litellm/pull/8950
* fix(azure.py): track azure llm api latency metric by krrishdholakia in https://github.com/BerriAI/litellm/pull/9217
* Support bedrock converse cache token tracking by krrishdholakia in https://github.com/BerriAI/litellm/pull/9221
* Emit audit logs on All user + model Create/Update/Delete endpoints by krrishdholakia in https://github.com/BerriAI/litellm/pull/9223
* (UI Usage) - Allow clicking into Top Keys when showing users Top API Key by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9225
* [Feat] Add Snowflake Cortex to LiteLLM by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9222
* [Fixes] Responses API - allow /responses and subpaths as LLM API route + Add exception mapping for responses API by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9220
* docs: Add centralized credential management docs by bexelbie in https://github.com/BerriAI/litellm/pull/9254
* Docs: Update configs.md by bexelbie in https://github.com/BerriAI/litellm/pull/9263
* Support reusing existing model credentials by krrishdholakia in https://github.com/BerriAI/litellm/pull/9267
* LiteLLM UI Fixes by krrishdholakia in https://github.com/BerriAI/litellm/pull/9269
* Fix "system" role has become unacceptable in ollama by briandevvn in https://github.com/BerriAI/litellm/pull/9261
* Litellm rc 03 14 2025 patch 1 by krrishdholakia in https://github.com/BerriAI/litellm/pull/9271
* [Feat] UI - Add Test Connection by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9272
* [UI] Fix 1 - instantly show newly create keys on Admin UI (don't require refresh) by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9257
* (UI) Fix model edit + delete - instantly show edit + deletes to models by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9258
New Contributors
* lucasra1 made their first contribution in https://github.com/BerriAI/litellm/pull/9180
* youngchannelforyou made their first contribution in https://github.com/BerriAI/litellm/pull/9193
* xucailiang made their first contribution in https://github.com/BerriAI/litellm/pull/8741
* SunnyWan59 made their first contribution in https://github.com/BerriAI/litellm/pull/8950
* bexelbie made their first contribution in https://github.com/BerriAI/litellm/pull/9254
* briandevvn made their first contribution in https://github.com/BerriAI/litellm/pull/9261
**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.63.8-nightly...v1.63.11-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.63.11-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.63.11-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.63.11-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.63.11-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.63.11-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.63.11-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 250.0 | 265.0124223657352 | 6.224465343838535 | 0.0 | 1862 | 0 | 213.4414930000048 | 2393.6349599999858 |
| Aggregated | Passed ✅ | 250.0 | 265.0124223657352 | 6.224465343838535 | 0.0 | 1862 | 0 | 213.4414930000048 | 2393.6349599999858 |
v1.63.8-nightly
What's Changed
* Delegate router azure client init logic to azure provider by krrishdholakia in https://github.com/BerriAI/litellm/pull/9140
* Bing Search Pass Thru by sfarthin in https://github.com/BerriAI/litellm/pull/8019
* [Feat] Add OpenAI Responses API to litellm python SDK by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9155
* Support credential management on Proxy - via CRUD endpoints - `credentials/*` by krrishdholakia in https://github.com/BerriAI/litellm/pull/9124
* Bump babel/runtime-corejs3 from 7.26.0 to 7.26.10 in /docs/my-website by dependabot in https://github.com/BerriAI/litellm/pull/9167
* Bump babel/helpers from 7.26.0 to 7.26.10 in /docs/my-website by dependabot in https://github.com/BerriAI/litellm/pull/9168
* fix(azure): Patch for Function Calling Bug & Update Default API Version to `2025-02-01-preview` by colesmcintosh in https://github.com/BerriAI/litellm/pull/9191
* [Feat] - Add Responses API on LiteLLM Proxy by ishaan-jaff in https://github.com/BerriAI/litellm/pull/9183
New Contributors
* sfarthin made their first contribution in https://github.com/BerriAI/litellm/pull/8019
**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.63.7-nightly...v1.63.8-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.63.8-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 230.0 | 258.44225107871796 | 6.155259588605481 | 0.0033416175833905974 | 1842 | 1 | 86.54871299995648 | 3971.497738000039 |
| Aggregated | Passed ✅ | 230.0 | 258.44225107871796 | 6.155259588605481 | 0.0033416175833905974 | 1842 | 1 | 86.54871299995648 | 3971.497738000039 |
v1.63.7-nightly
What's Changed
* add support for Amazon Nova Canvas model by omrishiv in https://github.com/BerriAI/litellm/pull/7838
* feat: add bedrock deepseek r1 model pricing by kearnsw in https://github.com/BerriAI/litellm/pull/9108
* fix(internal_user_endpoints.py): allow internal user to query their o… by krrishdholakia in https://github.com/BerriAI/litellm/pull/9162
* add support for Amazon Nova Canvas model (7838) by krrishdholakia in https://github.com/BerriAI/litellm/pull/9101
* Fix bedrock chunk parsing + azure whisper cost tracking by krrishdholakia in https://github.com/BerriAI/litellm/pull/9166
New Contributors
* omrishiv made their first contribution in https://github.com/BerriAI/litellm/pull/7838
* kearnsw made their first contribution in https://github.com/BerriAI/litellm/pull/9108
**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.63.6.dev1...v1.63.7-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.63.7-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 190.0 | 210.2989654185558 | 6.340335220104276 | 0.0 | 1897 | 0 | 164.78329899996425 | 3864.2000990000156 |
| Aggregated | Passed ✅ | 190.0 | 210.2989654185558 | 6.340335220104276 | 0.0 | 1897 | 0 | 164.78329899996425 | 3864.2000990000156 |