Litellm

What's Changed
* add langfuse logging by maxdeichmann in https://github.com/BerriAI/litellm/pull/2685
* `cryptography` and `python-multipart` version bump by RoniGurvich in https://github.com/BerriAI/litellm/pull/2672
* (feat) Perf Improvement - stop eagerly evaluating fstring by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2683
* Fix pip install 'litellm[extra_proxy]' by dragosMC91 in https://github.com/BerriAI/litellm/pull/2660


**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.34.0...v1.34.1

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 39 | 44.70487909533914 | 1.5764456770634518 | 0.0 | 472 | 0 | 35.648874999992586 | 767.226882999978 |
| /health/liveliness | Passed ✅ | 25 | 28.792333194952395 | 15.88135422550151 | 0.0 | 4755 | 0 | 23.25843599999189 | 1419.5746369999824 |
| /health/readiness | Passed ✅ | 25 | 28.926927010092044 | 15.223388550964435 | 0.0033399272819140927 | 4558 | 1 | 23.396136000030765 | 1421.804757000018 |
| Aggregated | Passed ✅ | 25 | 29.622604045682124 | 32.681188453529394 | 0.0033399272819140927 | 9785 | 1 | 23.25843599999189 | 1421.804757000018 |

**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.33.9...v1.34.0

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 38 | 42.04039730334579 | 1.596504378868726 | 0.0 | 478 | 0 | 35.187763999942945 | 524.0036399999326 |
| /health/liveliness | Passed ✅ | 25 | 27.013957424801724 | 15.567587677629984 | 0.0 | 4661 | 0 | 23.172639999984312 | 896.960622999984 |
| /health/readiness | Passed ✅ | 25 | 27.542442462175806 | 15.717886207021392 | 0.0 | 4706 | 0 | 23.277541000027213 | 1057.5831259999973 |
| Aggregated | Passed ✅ | 25 | 27.996150299136485 | 32.881978263520104 | 0.0 | 9845 | 0 | 23.172639999984312 | 1057.5831259999973 |

What's Changed
* [Docs] update gunicorn instructions - Uvicorn perf is significantly better on K8s by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2667
* [Docs] Add Docs on deploying to EKS Cluster + K8 by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2668
* feat(router.py): enable pre-call checks by krrishdholakia in https://github.com/BerriAI/litellm/pull/2669
* (docs) best prod practices by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2670
* fix(proxy_server.py): fix model info check by krrishdholakia in https://github.com/BerriAI/litellm/pull/2666


**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.33.8...v1.33.9

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 54 | 60.228549854209994 | 1.6267984265796072 | 0.0 | 487 | 0 | 49.45159799999033 | 794.4530399999792 |
| /health/liveliness | Passed ✅ | 40 | 43.17110106318068 | 15.650001290606694 | 0.0 | 4685 | 0 | 38.61002399997915 | 1231.5367759999845 |
| /health/readiness | Passed ✅ | 40 | 42.72228626508686 | 15.222423880745934 | 0.0 | 4557 | 0 | 38.460548000017525 | 1388.6105939999993 |
| Aggregated | Passed ✅ | 40 | 43.814715877274416 | 32.499223597932236 | 0.0 | 9729 | 0 | 38.460548000017525 | 1388.6105939999993 |

What's Changed
* Athina docs updated with information about additional fields and a minor fix in the callback by vivek-athina in https://github.com/BerriAI/litellm/pull/2659


**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.33.7...v1.33.8

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 78 | 83.18319155529923 | 1.4500632067856818 | 0.0 | 434 | 0 | 75.45461099999784 | 865.6005609999511 |
| /health/liveliness | Passed ✅ | 65 | 68.45390179128916 | 15.112064249519905 | 0.0 | 4523 | 0 | 63.496340999961376 | 1392.5121589999776 |
| /health/readiness | Passed ✅ | 65 | 67.99550030012846 | 15.529709182349883 | 0.0 | 4648 | 0 | 63.460964000000786 | 1175.6870899999967 |
| Aggregated | Passed ✅ | 65 | 68.89761461030682 | 32.09183663865547 | 0.0 | 9605 | 0 | 63.460964000000786 | 1392.5121589999776 |

What's Changed
* (feat) bump to python 3.11 - Improve Proxy perf 7% by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2646
* feat(handle_jwt.py): enable jwt-project based auth by krrishdholakia in https://github.com/BerriAI/litellm/pull/2650
* [FEAT] Add example Kubernetes + Service YAML Files by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2653
* (feat) remove litellm.telemetry - improve perf on EC2 tiny machines by 90% by ishaan-jaff in https://github.com/BerriAI/litellm/pull/2657


**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.33.4...v1.33.7



Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 79 | 90.82509035211159 | 1.422902757757835 | 0.0033401473186803637 | 426 | 1 | 68.13304000002063 | 1320.544376999976 |
| /health/liveliness | Passed ✅ | 65 | 67.60595521311457 | 15.484922969402167 | 0.0 | 4636 | 0 | 63.423195999973814 | 1027.6299029999905 |
| /health/readiness | Passed ✅ | 65 | 67.28953461176305 | 15.391398844479117 | 0.0 | 4608 | 0 | 63.412283999980446 | 1321.834434999971 |
| Aggregated | Passed ✅ | 65 | 68.47806332461236 | 32.299224571639115 | 0.0033401473186803637 | 9670 | 1 | 63.412283999980446 | 1321.834434999971 |

**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.33.3...v1.33.4

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Failed ❌ | 85 | 93.98301701746783 | 1.5302020467054798 | 0.1403242051563977 | 458 | 42 | 63.60682199999701 | 802.5505589999966 |
| /health/liveliness | Failed ❌ | 66 | 69.3704888863688 | 15.318725729573416 | 1.867648349581579 | 4585 | 559 | 63.33905499997172 | 1172.7252800000088 |
| /health/readiness | Failed ❌ | 66 | 69.7621130132432 | 15.388887832151616 | 1.7373473019363526 | 4606 | 520 | 63.29959000004237 | 1177.4286550000284 |
| Aggregated | Failed ❌ | 66 | 70.7256923906102 | 32.23781560843051 | 3.745319856674329 | 9649 | 1121 | 63.29959000004237 | 1177.4286550000284 |