What's Changed
* fix(get_litellm_params.py): handle no-log being passed in via kwargs by krrishdholakia in https://github.com/BerriAI/litellm/pull/8830
* fix(o_series_transformation.py): fix optional param check for o-serie… by krrishdholakia in https://github.com/BerriAI/litellm/pull/8787
* chore: set ttlSecondsAfterFinished on the migration job in the litellm-helm chart by ashwin153 in https://github.com/BerriAI/litellm/pull/8593
New Contributors
* ashwin153 made their first contribution in https://github.com/BerriAI/litellm/pull/8593
**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.61.17-nightly...v1.61.17.dev1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.17.dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Failed ❌ | 170.0 | 197.91492646891462 | 6.342832402753434 | 6.342832402753434 | 1898 | 1898 | 138.25161099998695 | 4268.093897000028 |
| Aggregated | Failed ❌ | 170.0 | 197.91492646891462 | 6.342832402753434 | 6.342832402753434 | 1898 | 1898 | 138.25161099998695 | 4268.093897000028 |
v1.61.17-nightly
What's Changed
* (UI) Fixes for managing Internal Users by ishaan-jaff in https://github.com/BerriAI/litellm/pull/8786
* Litellm contributor prs 02 24 2025 by krrishdholakia in https://github.com/BerriAI/litellm/pull/8781
* (UI) Allow adding MSFT SSO on UI by ishaan-jaff in https://github.com/BerriAI/litellm/pull/8779
* (UI) Minor fix, clear new team form after adding a new team by ishaan-jaff in https://github.com/BerriAI/litellm/pull/8776
* [ui] Icons on navbar profile dropdown by Mte90 in https://github.com/BerriAI/litellm/pull/8792
* fix(UI): model name overflow in model hub cards by Aditya-A-G in https://github.com/BerriAI/litellm/pull/8749
* fix vertex_ai claude 3.7 naming by emerzon in https://github.com/BerriAI/litellm/pull/8807
* (Router) - If `allowed_fails` or `allowed_fail_policy` set, use that for single deployment cooldown logic by ishaan-jaff in https://github.com/BerriAI/litellm/pull/8668
* (Bug fix) - reading /parsing request body when on hypercorn by ishaan-jaff in https://github.com/BerriAI/litellm/pull/8734
* (Bug fix) - running litellm proxy on wndows by ishaan-jaff in https://github.com/BerriAI/litellm/pull/8735
* Anthropic model cost map updates by krrishdholakia in https://github.com/BerriAI/litellm/pull/8816
* Adding Azure Phi-4 by emerzon in https://github.com/BerriAI/litellm/pull/8808
* (Bug Fix) Using LiteLLM Python SDK with model=`litellm_proxy/` for embedding, image_generation, transcription, speech, rerank by ishaan-jaff in https://github.com/BerriAI/litellm/pull/8815
* (Bug fix) - allow using Assistants GET, DELETE on `/openai` pass through routes by ishaan-jaff in https://github.com/BerriAI/litellm/pull/8818
* (Bug fix) dd-trace used by default on litellm proxy by ishaan-jaff in https://github.com/BerriAI/litellm/pull/8817
* Adding openrouter claude-3.7-sonnet by fengjiajie in https://github.com/BerriAI/litellm/pull/8826
* (UI) - Create Key flow for existing users by ishaan-jaff in https://github.com/BerriAI/litellm/pull/8844
New Contributors
* Mte90 made their first contribution in https://github.com/BerriAI/litellm/pull/8792
* Aditya-A-G made their first contribution in https://github.com/BerriAI/litellm/pull/8749
**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.61.16-nightly...v1.61.17-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.17-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.17-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Failed ❌ | 110.0 | 132.37492176765105 | 6.343856763236461 | 6.343856763236461 | 1898 | 1898 | 93.89094700003398 | 3315.9179240000185 |
| Aggregated | Failed ❌ | 110.0 | 132.37492176765105 | 6.343856763236461 | 6.343856763236461 | 1898 | 1898 | 93.89094700003398 | 3315.9179240000185 |
v1.61.16-nightly
What's Changed
* fix: remove aws params from bedrock embedding request body (8618) by ishaan-jaff in https://github.com/BerriAI/litellm/pull/8696
* Add anthropic3-7-sonnet by dragosMC91 in https://github.com/BerriAI/litellm/pull/8766
* fix incorrect variable name in reliability section of docs by niinpatel in https://github.com/BerriAI/litellm/pull/8753
* Litellm contributor prs 02 24 2025 by krrishdholakia in https://github.com/BerriAI/litellm/pull/8775
* Add anthropic thinking + reasoning content support by krrishdholakia in https://github.com/BerriAI/litellm/pull/8778
New Contributors
* niinpatel made their first contribution in https://github.com/BerriAI/litellm/pull/8753
**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.61.15-nightly...v1.61.16-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.16-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Failed ❌ | 150.0 | 173.09079325414655 | 6.248259928705226 | 6.248259928705226 | 1869 | 1869 | 131.67032300003711 | 1529.8640780000028 |
| Aggregated | Failed ❌ | 150.0 | 173.09079325414655 | 6.248259928705226 | 6.248259928705226 | 1869 | 1869 | 131.67032300003711 | 1529.8640780000028 |
v1.61.15-nightly
What's Changed
* Add cost tracking for rerank via bedrock + jina ai by krrishdholakia in https://github.com/BerriAI/litellm/pull/8691
* add bedrock llama vision support + cohere / infinity rerank - 'return_documents' support by krrishdholakia in https://github.com/BerriAI/litellm/pull/8684
* Correct spelling in user_management_heirarchy.md by oaustegard in https://github.com/BerriAI/litellm/pull/8716
* (Feat) - UI, Allow sorting models by Created_At and all other columns on the UI by ishaan-jaff in https://github.com/BerriAI/litellm/pull/8725
* (UI) Edit Model flow improvements by ishaan-jaff in https://github.com/BerriAI/litellm/pull/8729
* Support arize phoenix on litellm proxy (7756) by krrishdholakia in https://github.com/BerriAI/litellm/pull/8715
* fix(amazon_deepseek_transformation.py): remove </think> from stream o… by krrishdholakia in https://github.com/BerriAI/litellm/pull/8717
* Add cohere v2/rerank support (8421) by krrishdholakia in https://github.com/BerriAI/litellm/pull/8605
* fix(proxy/_types.py): fixes issue where internal user able to escalat… by krrishdholakia in https://github.com/BerriAI/litellm/pull/8740
New Contributors
* oaustegard made their first contribution in https://github.com/BerriAI/litellm/pull/8716
**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.61.13-nightly...v1.61.15-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.15-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Failed ❌ | 140.0 | 149.21232461729608 | 6.452882828983183 | 6.452882828983183 | 1931 | 1931 | 114.22628599996187 | 662.278525000005 |
| Aggregated | Failed ❌ | 140.0 | 149.21232461729608 | 6.452882828983183 | 6.452882828983183 | 1931 | 1931 | 114.22628599996187 | 662.278525000005 |
v1.61.13-stable
**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.61.13-nightly...v1.61.13-stable
🚨 Known Issue:
- DD Trace was enabled by default on litellm docker: https://github.com/BerriAI/litellm/issues/8788
- Expect a patched v1.61.13-stable with the fix
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.61.13-stable
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Failed ❌ | 180.0 | 204.35323738803527 | 6.313934995711798 | 6.313934995711798 | 1889 | 1889 | 146.12962300003574 | 2180.2391240000247 |
| Aggregated | Failed ❌ | 180.0 | 204.35323738803527 | 6.313934995711798 | 6.313934995711798 | 1889 | 1889 | 146.12962300003574 | 2180.2391240000247 |
v1.55.8-stable-patched
**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.55.8-stable...v1.55.8-stable-patched
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm_stable_release_branch-v1.55.8-stable-patched
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Failed ❌ | 150.0 | 172.39167931961987 | 6.334948735889217 | 6.334948735889217 | 1896 | 1896 | 131.01931900001773 | 2316.6445349999663 |
| Aggregated | Failed ❌ | 150.0 | 172.39167931961987 | 6.334948735889217 | 6.334948735889217 | 1896 | 1896 | 131.01931900001773 | 2316.6445349999663 |