Litellm

What's Changed
* ui - pass extra litellm params by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2900


**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.34.34.dev1...v1.34.35



Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 91 | 96.13553076973712 | 1.5233025728555512 | 0.0 | 456 | 0 | 86.62122199996247 | 482.4095380000131 |
| /health/liveliness | Passed ✅ | 76 | 79.2783738783157 | 15.236366304373178 | 0.0 | 4561 | 0 | 74.1453349999972 | 1484.2781819999686 |
| /health/readiness | Passed ✅ | 76 | 79.9449014786066 | 15.537018127963089 | 0.0 | 4651 | 0 | 74.1601340000102 | 1323.9347329999873 |
| Aggregated | Passed ✅ | 76 | 80.39410447527911 | 32.296687005191814 | 0.0 | 9668 | 0 | 74.1453349999972 | 1484.2781819999686 |

1.34.35-stable
**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.34.36.dev2...1.34.35-stable



Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 82 | 85.9205370193538 | 1.5532033800118876 | 0.0 | 465 | 0 | 75.86904800001548 | 290.69797300002165 |
| /health/liveliness | Passed ✅ | 65 | 68.7640741313527 | 15.511992466183239 | 0.0033402223226062103 | 4644 | 1 | 63.21401899998591 | 1120.7676690000312 |
| /health/readiness | Passed ✅ | 65 | 68.20540649619821 | 15.375043350956386 | 0.0033402223226062103 | 4603 | 1 | 63.38805799998681 | 1236.8685650000089 |
| Aggregated | Passed ✅ | 65 | 69.32072653233155 | 32.44023919715151 | 0.006680444645212421 | 9712 | 2 | 63.21401899998591 | 1236.8685650000089 |

What's Changed
* docs fix gpt-3.5-turbo-instruct-0914 by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2882
* [Feat] Async Anthropic API 97.5% lower median latency by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2879


**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.34.33...v1.34.34

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 77 | 81.2559030586862 | 1.4229723751978038 | 0.0 | 426 | 0 | 71.93818000001784 | 594.9944120000055 |
| /health/liveliness | Passed ✅ | 62 | 64.5417521727825 | 15.29194256726654 | 0.0 | 4578 | 0 | 59.87314999998716 | 1318.4233759999984 |
| /health/readiness | Passed ✅ | 62 | 64.09058208715463 | 15.445596861301983 | 0.0 | 4624 | 0 | 59.911668999973244 | 1176.3459180000382 |
| Aggregated | Passed ✅ | 62 | 65.06460404248075 | 32.160511803766326 | 0.0 | 9628 | 0 | 59.87314999998716 | 1318.4233759999984 |

**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.34.34...v1.34.34.dev1

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 40 | 44.63567630967619 | 1.5531698625084285 | 0.0 | 465 | 0 | 34.3844149999768 | 947.8966600000263 |
| /health/liveliness | Passed ✅ | 25 | 28.492204595294115 | 15.615202381133125 | 0.0 | 4675 | 0 | 23.249706000001424 | 1367.5765010000305 |
| /health/readiness | Passed ✅ | 25 | 28.254767451092132 | 15.742128090327363 | 0.0033401502419536096 | 4713 | 1 | 23.426859999972294 | 1369.3040239999732 |
| Aggregated | Passed ✅ | 25 | 29.140501873946636 | 32.91050033396891 | 0.0033401502419536096 | 9853 | 1 | 23.249706000001424 | 1369.3040239999732 |

What's Changed
* feat(proxy/utils.py): return api base for request hanging alerts by krrishdholakia in https://github.com/BerriAI/litellm/pull/2880


**Full Changelog**: https://github.com/BerriAI/litellm/compare/1.34.2...v1.34.33

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 93 | 96.96510066880384 | 1.5634829556893368 | 0.0 | 468 | 0 | 86.43016700000317 | 463.10457399999905 |
| /health/liveliness | Passed ✅ | 77 | 80.00998962632444 | 15.457768452937097 | 0.0 | 4627 | 0 | 73.85146199999326 | 1299.2101319999847 |
| /health/readiness | Passed ✅ | 77 | 80.26552238425697 | 15.066897714014763 | 0.0 | 4510 | 0 | 73.70257700000593 | 1044.845262000024 |
| Aggregated | Passed ✅ | 77 | 80.95610568110362 | 32.0881491226412 | 0.0 | 9605 | 0 | 73.70257700000593 | 1299.2101319999847 |

What's Changed
* Support latest Anthropic Tools Use (2024-04-04) by lazyhope in https://github.com/BerriAI/litellm/pull/2856
* [FEAT] add azure command-r-plus by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2861
https://docs.litellm.ai/docs/providers/azure_ai
* Add Azure Command-r-plus on litellm proxy by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2868
* [CI/CD] Use specific commit hash on docker builds by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2869
* CI/CD deploy stable releases by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2871
* feat(prisma.schema): UI improvements for tracking spend for users by krrishdholakia in https://github.com/BerriAI/litellm/pull/2870


**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.34.28.dev12...v1.34.29

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 93 | 109.58477577729985 | 1.5598872259349816 | 0.0 | 467 | 0 | 86.14991300001407 | 1266.8051839999919 |
| /health/liveliness | Passed ✅ | 78 | 79.6745395084596 | 15.201384936252895 | 0.003340229605856492 | 4551 | 1 | 73.84244000002127 | 1081.535916000007 |
| /health/readiness | Passed ✅ | 78 | 80.26678459095872 | 15.294911365216876 | 0.0 | 4579 | 0 | 73.68360599997459 | 1256.5205770000034 |
| Aggregated | Passed ✅ | 78 | 81.41257958038933 | 32.05618352740475 | 0.003340229605856492 | 9597 | 1 | 73.68360599997459 | 1266.8051839999919 |

What's Changed
* Add command-r-plus by themrzmaster in https://github.com/BerriAI/litellm/pull/2847
* Fix: Remove non-existent gemini-1.5-pro-vision model. by Manouchehri in https://github.com/BerriAI/litellm/pull/2841
* feat(vertex_ai_anthropic.py): Add support for claude 3 on vertex ai by krrishdholakia in https://github.com/BerriAI/litellm/pull/2665
* feat: add gemini-experimental to vertex ai by cheahjs in https://github.com/BerriAI/litellm/pull/2859
* Support latest Anthropic Tools Use (2024-04-04) by lazyhope in https://github.com/BerriAI/litellm/pull/2856

New Contributors
* themrzmaster made their first contribution in https://github.com/BerriAI/litellm/pull/2847
* cheahjs made their first contribution in https://github.com/BerriAI/litellm/pull/2859

**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.34.27...v1.34.28













Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 40 | 53.3972479797162 | 1.6466886940720966 | 0.0 | 493 | 0 | 35.56440799999905 | 1050.790050000046 |
| /health/liveliness | Passed ✅ | 25 | 27.134720360847133 | 15.458164860376598 | 0.0 | 4628 | 0 | 23.121685999967667 | 1016.2047730000268 |
| /health/readiness | Passed ✅ | 25 | 26.557904771140425 | 15.879022417076566 | 0.0 | 4754 | 0 | 23.183289999963108 | 737.5355639999839 |
| Aggregated | Passed ✅ | 25 | 28.168162872506542 | 32.983875971525265 | 0.0 | 9875 | 0 | 23.121685999967667 | 1050.790050000046 |