Litellm

**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.34.42...v1.35.0

[fix(caching.py): fix async batch redis get request](https://github.com/BerriAI/litellm/commit/76bd667b121305cb4a7a8d3633faedfff4f530f8) by krrishdholakia https://github.com/BerriAI/litellm/commit/76bd667b121305cb4a7a8d3633faedfff4f530f8

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 93 | 97.65305423059934 | 1.5068119533442674 | 0.0 | 451 | 0 | 86.16572899998687 | 1209.6832269999993 |
| /health/liveliness | Passed ✅ | 79 | 80.3909614859205 | 15.305333832084456 | 0.0 | 4581 | 0 | 74.10348499999486 | 759.2023230000109 |
| /health/readiness | Passed ✅ | 79 | 80.52192505215501 | 15.502455573209316 | 0.0 | 4640 | 0 | 74.02658799998107 | 967.6750630000015 |
| Aggregated | Passed ✅ | 79 | 81.25871115250257 | 32.314601358638036 | 0.0 | 9672 | 0 | 74.02658799998107 | 1209.6832269999993 |

What's Changed
* ui - setup sso on admin ui 👩‍🚀 by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2934
* [UI] Fix SSO Error - user not allowed to access info by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2939
* Fix - Embedding Caching - allow base 64 embedding cache hits by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2943
* Feature: add gemini-1.0-pro-001 and gemini-1.0-pro-002 to models list by Manouchehri in https://github.com/BerriAI/litellm/pull/2940
* fix - return stack trace on failing /health checks - first 1000 chars by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2923
* fix - /config/update setting env variables by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2946
* Router Async Improvements by krrishdholakia in https://github.com/BerriAI/litellm/pull/2942
* [FIX] - Security issue Server-Side Template Injection in /completions endpoint in by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2941


**Full Changelog**: https://github.com/BerriAI/litellm/compare/1.34.40.dev1...v1.34.42

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 41 | 45.62665910765985 | 1.6134726654854352 | 0.0 | 483 | 0 | 35.47842400001855 | 1423.002457999985 |
| /health/liveliness | Passed ✅ | 25 | 27.266815498923485 | 15.506708308868303 | 0.0 | 4642 | 0 | 23.460693000004085 | 1011.4049249999653 |
| /health/readiness | Passed ✅ | 25 | 28.117354577288825 | 15.797333820042699 | 0.0 | 4729 | 0 | 23.34315000001652 | 1129.9532500000282 |
| Aggregated | Passed ✅ | 25 | 28.574914115181734 | 32.91751479439644 | 0.0 | 9854 | 0 | 23.34315000001652 | 1423.002457999985 |

What's Changed
* Fix - Embedding Caching - allow base 64 embedding cache hits by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2943


**Full Changelog**: https://github.com/BerriAI/litellm/compare/1.34.39.dev1...v1.34.41

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 55 | 60.27472656762199 | 1.6302017598732303 | 0.0 | 488 | 0 | 49.61538999998538 | 853.5629180000228 |
| /health/liveliness | Passed ✅ | 40 | 42.21640406246 | 15.617199236490475 | 0.0 | 4675 | 0 | 38.64845300000752 | 1119.598647000032 |
| /health/readiness | Passed ✅ | 41 | 43.405607340724444 | 15.216329951275746 | 0.0 | 4555 | 0 | 38.678258999993886 | 1239.126835000036 |
| Aggregated | Passed ✅ | 41 | 43.680623275776924 | 32.463730947639455 | 0.0 | 9718 | 0 | 38.64845300000752 | 1239.126835000036 |

What's Changed
* ui - setup sso on admin ui 👩‍🚀 by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2934
* [UI] Fix SSO Error - user not allowed to access info by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2939
* ui - runtime error fix by krrishdholakia


**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.34.39...v1.34.40

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 55 | 59.38243058163271 | 1.636716795176527 | 0.0 | 490 | 0 | 50.21895799995946 | 823.0557899999837 |
| /health/liveliness | Passed ✅ | 40 | 43.38840214425958 | 15.652356943259601 | 0.003340238357503116 | 4686 | 1 | 38.32955799998672 | 1505.0786130000233 |
| /health/readiness | Passed ✅ | 40 | 43.678412320061014 | 15.351735491084321 | 0.0 | 4596 | 0 | 38.477973999988535 | 1445.2507309999874 |
| Aggregated | Passed ✅ | 40 | 44.32679353827271 | 32.64080922952045 | 0.003340238357503116 | 9772 | 1 | 38.32955799998672 | 1505.0786130000233 |

**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.34.41...1.34.40.dev1

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 77 | 81.173657764966 | 1.5063339924614734 | 0.0 | 451 | 0 | 71.93741999998338 | 766.846145000045 |
| /health/liveliness | Passed ✅ | 62 | 65.36667029825317 | 15.487518232913198 | 0.003339986679515462 | 4637 | 1 | 59.801646999972036 | 1681.3968060000093 |
| /health/readiness | Passed ✅ | 62 | 66.13317987362252 | 15.460798339477074 | 0.0 | 4629 | 0 | 59.8357880000151 | 1439.9600270000406 |
| Aggregated | Passed ✅ | 62 | 66.4654790017493 | 32.45465056485175 | 0.003339986679515462 | 9717 | 1 | 59.801646999972036 | 1681.3968060000093 |

🚨 UI Runtime error. Wait till next release to upgrade, if you use LiteLLM UI

What's Changed
* Added support for newly released `gpt-4-turbo-2024-04-09` and `gpt-4-turbo` by ventz in https://github.com/BerriAI/litellm/pull/2920
* feat: add gemini-1.5-pro-preview-0409 to vertex ai by cheahjs in https://github.com/BerriAI/litellm/pull/2914
* feat(gemini.py): support google-genai system instruction by krrishdholakia in https://github.com/BerriAI/litellm/pull/2925
* Gemini pro 1.5 model is now named correctly for AI Studio. by jpshackelford in https://github.com/BerriAI/litellm/pull/2926
* Clark/tpm rpm docs by CLARKBENHAM in https://github.com/BerriAI/litellm/pull/2909
* fix(main.py): handle translating text completion openai to chat completion for async requests by krrishdholakia in https://github.com/BerriAI/litellm/pull/2924

New Contributors
* ventz made their first contribution in https://github.com/BerriAI/litellm/pull/2920
* jpshackelford made their first contribution in https://github.com/BerriAI/litellm/pull/2926

**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.34.38...v1.34.39

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 40 | 43.746598010869974 | 1.5368923320897772 | 0.0 | 460 | 0 | 35.04595399999744 | 468.7995090000072 |
| /health/liveliness | Passed ✅ | 25 | 27.79845553243649 | 15.398992953482136 | 0.0 | 4609 | 0 | 23.410484999999426 | 1118.1426470000133 |
| /health/readiness | Passed ✅ | 25 | 27.354294543427137 | 15.579410748988327 | 0.0 | 4663 | 0 | 23.39587599999504 | 1305.677650000007 |
| Aggregated | Passed ✅ | 25 | 28.33945664714355 | 32.515296034560244 | 0.0 | 9732 | 0 | 23.39587599999504 | 1305.677650000007 |