Litellm

**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.35.28.dev1...v1.35.29

Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 42 | 49.42966661123069 | 1.5463498980603823 | 0.0 | 463 | 0 | 35.05075499998611 | 750.6745339999839 |
| /health/liveliness | Passed ✅ | 25 | 28.80921247507392 | 15.810843234163821 | 0.0 | 4734 | 0 | 23.197104999979956 | 1141.1978890000114 |
| /health/readiness | Passed ✅ | 26 | 28.33507187739932 | 15.30986594537536 | 0.0 | 4584 | 0 | 23.20666399998572 | 1207.45605999997 |
| Aggregated | Passed ✅ | 26 | 29.563103668745402 | 32.66705907759956 | 0.0 | 9781 | 0 | 23.197104999979956 | 1207.45605999997 |

What's Changed
* [UI] View Number of Keys, Members in a team by ishaan-jaff in https://github.com/BerriAI/litellm/pull/3300
* fix Llama models messages to prompt conversion for AWS Bedrock by aswny in https://github.com/BerriAI/litellm/pull/3298
* [UI] Show Tooltip when editing models by ishaan-jaff in https://github.com/BerriAI/litellm/pull/3296
* [UI] fix end users tab by ishaan-jaff in https://github.com/BerriAI/litellm/pull/3306
* [Backend-Alerting] Separate alerting for different channels by ishaan-jaff in https://github.com/BerriAI/litellm/pull/3307
* [FEAT - UI + Backend] Proxy - set unique webhooks per alert by ishaan-jaff in https://github.com/BerriAI/litellm/pull/3309
* (requirements.txt) - Update gunicorn to 22.0.0 for CVE-2024-1135 by Manouchehri in https://github.com/BerriAI/litellm/pull/3265
* build(deps): bump hono from 4.1.5 to 4.2.7 in /litellm-js/spend-logs by dependabot in https://github.com/BerriAI/litellm/pull/3239

New Contributors
* aswny made their first contribution in https://github.com/BerriAI/litellm/pull/3298

**Full Changelog**: https://github.com/BerriAI/litellm/compare/1.35.26.dev3...v1.35.28

Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 83 | 90.47377557849362 | 1.553283816541804 | 0.0 | 465 | 0 | 76.9804239999985 | 1407.7624669999977 |
| /health/liveliness | Passed ✅ | 66 | 68.11714478811162 | 15.22886219271846 | 0.0 | 4559 | 0 | 63.024813999959406 | 764.0456139999969 |
| /health/readiness | Passed ✅ | 66 | 69.1427847725483 | 15.258925750457978 | 0.0 | 4568 | 0 | 63.323490000016136 | 1336.527273999991 |
| Aggregated | Passed ✅ | 66 | 69.68938798728121 | 32.04107175971824 | 0.0 | 9592 | 0 | 63.024813999959406 | 1407.7624669999977 |

What's Changed
* fix(utils.py): fix the response object returned when n>1 for stream=true by krrishdholakia in https://github.com/BerriAI/litellm/pull/3308
* [Fix] sending deployment latencies to slack alerting - lowest_latency by ishaan-jaff in https://github.com/BerriAI/litellm/pull/3301
* fix(proxy/utils.py): log rejected proxy requests to langfuse by krrishdholakia in https://github.com/BerriAI/litellm/pull/3310


**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.35.28...v1.35.28.dev1

Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 78 | 85.02681108551172 | 1.4062802929202902 | 0.0 | 421 | 0 | 72.50512399997433 | 1238.4483960000239 |
| /health/liveliness | Passed ✅ | 62 | 64.63901720184093 | 15.241940561984046 | 0.0033403332373403566 | 4563 | 1 | 59.40863600000057 | 1528.008194999984 |
| /health/readiness | Passed ✅ | 62 | 64.47801373834994 | 15.626078884278188 | 0.0 | 4678 | 0 | 59.54985199997509 | 1021.8157470000051 |
| Aggregated | Passed ✅ | 62 | 65.4494174318984 | 32.274299739182524 | 0.0033403332373403566 | 9662 | 1 | 59.40863600000057 | 1528.008194999984 |

What's Changed
* fix(utils.py): fix streaming to not return usage dict by krrishdholakia in https://github.com/BerriAI/litellm/pull/3267


**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.35.25...v1.35.26

Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 81 | 88.94697354090745 | 1.469613483633916 | 0.0 | 440 | 0 | 75.7317819999912 | 1133.430020999981 |
| /health/liveliness | Passed ✅ | 65 | 68.23953488817554 | 15.083578391115374 | 0.0 | 4516 | 0 | 63.34921199999144 | 1308.4431269999754 |
| /health/readiness | Passed ✅ | 66 | 68.74088326275232 | 15.584582987808755 | 0.0 | 4666 | 0 | 63.6032900000032 | 1463.0823009999858 |
| Aggregated | Passed ✅ | 66 | 69.42957485107057 | 32.13777486255805 | 0.0 | 9622 | 0 | 63.34921199999144 | 1463.0823009999858 |

What's Changed
* Fixed typo in token_usage.md by andeplane in https://github.com/BerriAI/litellm/pull/3289

New Contributors
* andeplane made their first contribution in https://github.com/BerriAI/litellm/pull/3289

**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.35.26.dev1...1.35.26.dev3

Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 82 | 89.29392581584473 | 1.5598100175990155 | 0.0 | 467 | 0 | 74.95227900000145 | 302.8643650000049 |
| /health/liveliness | Passed ✅ | 66 | 72.03278793199252 | 15.2741139410713 | 0.0 | 4573 | 0 | 63.32584099999394 | 913.2293100000197 |
| /health/readiness | Passed ✅ | 66 | 72.97123943137227 | 15.671581590095462 | 0.006680128555027903 | 4692 | 2 | 63.26687699998956 | 1357.0392919999676 |
| Aggregated | Passed ✅ | 66 | 73.31352835809702 | 32.505505548765775 | 0.006680128555027903 | 9732 | 2 | 63.26687699998956 | 1357.0392919999676 |

**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.35.26...v1.35.26.dev1

Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat



Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 94 | 103.60488741425448 | 1.4996401354988889 | 0.0 | 449 | 0 | 86.57567200000926 | 1088.8625389999902 |
| /health/liveliness | Passed ✅ | 78 | 82.48041119317014 | 15.647692727867026 | 0.0 | 4685 | 0 | 73.77187099996263 | 1435.7541210000022 |
| /health/readiness | Passed ✅ | 78 | 81.45476896904056 | 15.10327993925607 | 0.0033399557583494188 | 4522 | 1 | 73.75126800002363 | 1574.0074770000092 |
| Aggregated | Passed ✅ | 78 | 82.98237222110606 | 32.25061280262199 | 0.0033399557583494188 | 9656 | 1 | 73.75126800002363 | 1574.0074770000092 |