Litellm

What's Changed
* (fix) get_response_headers for Azure OpenAI by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6344
* fix(litellm-helm): correctly use dbReadyImage and dbReadyTag values by Hexoplon in https://github.com/BerriAI/litellm/pull/6336
* fix(proxy_server.py): add 'admin' user to db by krrishdholakia in https://github.com/BerriAI/litellm/pull/6223
* refactor(redis_cache.py): use a default cache value when writing to r… by krrishdholakia in https://github.com/BerriAI/litellm/pull/6358
* LiteLLM Minor Fixes & Improvements (10/21/2024) by krrishdholakia in https://github.com/BerriAI/litellm/pull/6352
* Refactor: apply early return by Haknt in https://github.com/BerriAI/litellm/pull/6369
* (refactor) remove berrispendLogger - unused logging integration by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6363
* (fix) standard logging metadata + add unit testing by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6366
* Revert "(fix) standard logging metadata + add unit testing " by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6381
* Add new Claude 3.5 sonnet model card by lowjiansheng in https://github.com/BerriAI/litellm/pull/6378
* Add claude 3 5 sonnet 20241022 models for all provides by Manouchehri in https://github.com/BerriAI/litellm/pull/6380

New Contributors
* Hexoplon made their first contribution in https://github.com/BerriAI/litellm/pull/6336
* Haknt made their first contribution in https://github.com/BerriAI/litellm/pull/6369

**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.50.1...v1.50.2



Docker Run LiteLLM Proxy


docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.50.2



Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 240.0 | 271.2844291307854 | 6.2111756488034775 | 0.0 | 1858 | 0 | 210.62568199999987 | 3226.4373430000433 |
| Aggregated | Passed ✅ | 240.0 | 271.2844291307854 | 6.2111756488034775 | 0.0 | 1858 | 0 | 210.62568199999987 | 3226.4373430000433 |

v1.50.1-stable.1
What's Changed
* fix(anthropic/chat/transformation.py): fix anthropic header [STABLE BRANCH] by krrishdholakia in https://github.com/BerriAI/litellm/pull/6365


**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.50.1...v1.50.1-stable.1



Docker Run LiteLLM Proxy


docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.50.1-stable.1



Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 190.0 | 209.56840744045144 | 6.231012229664211 | 0.0 | 1864 | 0 | 177.2575180000331 | 3604.4288230000348 |
| Aggregated | Passed ✅ | 190.0 | 209.56840744045144 | 6.231012229664211 | 0.0 | 1864 | 0 | 177.2575180000331 | 3604.4288230000348 |

What's Changed
* doc - using gpt-4o-audio-preview by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6326
* (refactor) `get_cache_key` to be under 100 LOC function by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6327
* Litellm openai audio streaming by krrishdholakia in https://github.com/BerriAI/litellm/pull/6325
* LiteLLM Minor Fixes & Improvements (10/18/2024) by krrishdholakia in https://github.com/BerriAI/litellm/pull/6320
* LiteLLM Minor Fixes & Improvements (10/19/2024) by krrishdholakia in https://github.com/BerriAI/litellm/pull/6331
* fix - unhandled jsonDecodeError in `convert_to_model_response_object` by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6338
* (testing) add test coverage for init custom logger class by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6341


**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.50.0...v1.50.1



Docker Run LiteLLM Proxy


docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.50.1



Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 260.0 | 288.9506471715694 | 6.1364168904754175 | 0.0 | 1836 | 0 | 231.4412910000101 | 1825.7555540000112 |
| Aggregated | Passed ✅ | 260.0 | 288.9506471715694 | 6.1364168904754175 | 0.0 | 1836 | 0 | 231.4412910000101 | 1825.7555540000112 |

v1.50.0-stable
What's Changed
* (feat) add `gpt-4o-audio-preview` models to model cost map by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6306
* (code quality) add ruff check PLR0915 for `too-many-statements` by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6309
* (doc) fix typo on Turn on / off caching per Key. by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6297
* (feat) Support `audio`, `modalities` params by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6304
* (feat) Support audio param in responses streaming by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6312
* (feat) - allow using os.environ/ vars for any value on config.yaml by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6276


**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.49.7...v1.50.0-stable



Docker Run LiteLLM Proxy


docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.50.0-stable



Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 250.0 | 280.1783744989076 | 6.121418649325928 | 0.0 | 1832 | 0 | 224.80250699993576 | 1589.2013160000715 |
| Aggregated | Passed ✅ | 250.0 | 280.1783744989076 | 6.121418649325928 | 0.0 | 1832 | 0 | 224.80250699993576 | 1589.2013160000715 |

**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.50.1...v1.50.1.dev4



Docker Run LiteLLM Proxy


docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.50.1.dev4



Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Failed ❌ | 270.0 | 318.21334593326156 | 6.115601652019426 | 0.0 | 1828 | 0 | 235.8845429999974 | 3021.9188690000465 |
| Aggregated | Failed ❌ | 270.0 | 318.21334593326156 | 6.115601652019426 | 0.0 | 1828 | 0 | 235.8845429999974 | 3021.9188690000465 |

v1.50.1-stable
What's Changed
* doc - using gpt-4o-audio-preview by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6326
* (refactor) `get_cache_key` to be under 100 LOC function by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6327
* Litellm openai audio streaming by krrishdholakia in https://github.com/BerriAI/litellm/pull/6325
* LiteLLM Minor Fixes & Improvements (10/18/2024) by krrishdholakia in https://github.com/BerriAI/litellm/pull/6320
* LiteLLM Minor Fixes & Improvements (10/19/2024) by krrishdholakia in https://github.com/BerriAI/litellm/pull/6331
* fix - unhandled jsonDecodeError in `convert_to_model_response_object` by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6338
* (testing) add test coverage for init custom logger class by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6341


**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.50.0...v1.50.1-stable



Docker Run LiteLLM Proxy


docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.50.1-stable



Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat





Docker Run LiteLLM Proxy


docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.50.1-stable



Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 270.0 | 297.1474404657832 | 6.054198370866852 | 0.0 | 1812 | 0 | 229.8348699999906 | 1982.3816379999926 |
| Aggregated | Passed ✅ | 270.0 | 297.1474404657832 | 6.054198370866852 | 0.0 | 1812 | 0 | 229.8348699999906 | 1982.3816379999926 |

What's Changed
* (fix) get_response_headers for Azure OpenAI by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6344
* fix(litellm-helm): correctly use dbReadyImage and dbReadyTag values by Hexoplon in https://github.com/BerriAI/litellm/pull/6336
* fix(proxy_server.py): add 'admin' user to db by krrishdholakia in https://github.com/BerriAI/litellm/pull/6223
* refactor(redis_cache.py): use a default cache value when writing to r… by krrishdholakia in https://github.com/BerriAI/litellm/pull/6358
* LiteLLM Minor Fixes & Improvements (10/21/2024) by krrishdholakia in https://github.com/BerriAI/litellm/pull/6352

New Contributors
* Hexoplon made their first contribution in https://github.com/BerriAI/litellm/pull/6336

**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.50.1...v1.50.1.dev1



Docker Run LiteLLM Proxy


docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.50.1.dev1



Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 200.0 | 220.3880747854055 | 6.181213384368117 | 0.0 | 1850 | 0 | 179.4118180000055 | 2854.2284040000254 |
| Aggregated | Passed ✅ | 200.0 | 220.3880747854055 | 6.181213384368117 | 0.0 | 1850 | 0 | 179.4118180000055 | 2854.2284040000254 |

v1.50.2-stable
What's Changed
* (fix) get_response_headers for Azure OpenAI by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6344
* fix(litellm-helm): correctly use dbReadyImage and dbReadyTag values by Hexoplon in https://github.com/BerriAI/litellm/pull/6336
* fix(proxy_server.py): add 'admin' user to db by krrishdholakia in https://github.com/BerriAI/litellm/pull/6223
* refactor(redis_cache.py): use a default cache value when writing to r… by krrishdholakia in https://github.com/BerriAI/litellm/pull/6358
* LiteLLM Minor Fixes & Improvements (10/21/2024) by krrishdholakia in https://github.com/BerriAI/litellm/pull/6352
* Refactor: apply early return by Haknt in https://github.com/BerriAI/litellm/pull/6369
* (refactor) remove berrispendLogger - unused logging integration by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6363
* (fix) standard logging metadata + add unit testing by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6366
* Revert "(fix) standard logging metadata + add unit testing " by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6381
* Add new Claude 3.5 sonnet model card by lowjiansheng in https://github.com/BerriAI/litellm/pull/6378
* Add claude 3 5 sonnet 20241022 models for all provides by Manouchehri in https://github.com/BerriAI/litellm/pull/6380

New Contributors
* Hexoplon made their first contribution in https://github.com/BerriAI/litellm/pull/6336
* Haknt made their first contribution in https://github.com/BerriAI/litellm/pull/6369

**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.50.1...v1.50.2-stable



Docker Run LiteLLM Proxy


docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.50.2-stable



Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat





Docker Run LiteLLM Proxy


docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.50.2-stable



Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 190.0 | 210.17145067557462 | 6.253172473880384 | 0.0 | 1871 | 0 | 177.3328190000143 | 1816.3144349999811 |
| Aggregated | Passed ✅ | 190.0 | 210.17145067557462 | 6.253172473880384 | 0.0 | 1871 | 0 | 177.3328190000143 | 1816.3144349999811 |

What's Changed
* (feat) add `gpt-4o-audio-preview` models to model cost map by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6306
* (code quality) add ruff check PLR0915 for `too-many-statements` by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6309
* (doc) fix typo on Turn on / off caching per Key. by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6297
* (feat) Support `audio`, `modalities` params by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6304
* (feat) Support audio param in responses streaming by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6312
* (feat) - allow using os.environ/ vars for any value on config.yaml by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6276


**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.49.7...v1.50.0



Docker Run LiteLLM Proxy


docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.50.0



Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 240.0 | 266.05337712404867 | 6.142852534799847 | 0.0 | 1838 | 0 | 211.22095199996238 | 1541.6589870000053 |
| Aggregated | Passed ✅ | 240.0 | 266.05337712404867 | 6.142852534799847 | 0.0 | 1838 | 0 | 211.22095199996238 | 1541.6589870000053 |

v1.49.7-stable
What's Changed
* Revert "(perf) move s3 logging to Batch logging + async [94% faster p… by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6275
* (testing) add test coverage for LLM OTEL logging by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6227
* (testing) add unit tests for LLMCachingHandler Class by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6279
* LiteLLM Minor Fixes & Improvements (10/17/2024) by krrishdholakia in https://github.com/BerriAI/litellm/pull/6293


**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.49.6...v1.49.7-stable



Docker Run LiteLLM Proxy


docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.49.7-stable



Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 250.0 | 275.82870433443276 | 6.089150330248114 | 0.0 | 1821 | 0 | 224.8554669999976 | 1500.5543909999801 |
| Aggregated | Passed ✅ | 250.0 | 275.82870433443276 | 6.089150330248114 | 0.0 | 1821 | 0 | 224.8554669999976 | 1500.5543909999801 |

What's Changed
* Revert "(perf) move s3 logging to Batch logging + async [94% faster p… by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6275
* (testing) add test coverage for LLM OTEL logging by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6227
* (testing) add unit tests for LLMCachingHandler Class by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6279
* LiteLLM Minor Fixes & Improvements (10/17/2024) by krrishdholakia in https://github.com/BerriAI/litellm/pull/6293


**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.49.6...v1.49.7



Docker Run LiteLLM Proxy


docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.49.7



Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Passed ✅ | 200.0 | 217.72068803611796 | 6.198611902745536 | 0.0 | 1855 | 0 | 176.8321219999507 | 1433.260539999992 |
| Aggregated | Passed ✅ | 200.0 | 217.72068803611796 | 6.198611902745536 | 0.0 | 1855 | 0 | 176.8321219999507 | 1433.260539999992 |

v1.49.6-stable
What's Changed
* (router testing) Add testing coverage for `run_async_fallback` and `run_sync_fallback` by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6256
* LiteLLM Minor Fixes & Improvements (10/15/2024) by krrishdholakia in https://github.com/BerriAI/litellm/pull/6242
* (testing) Router add testing coverage by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6253
* (testing) add router unit testing for `send_llm_exception_alert` , `router_cooldown_event_callback` , cooldown utils by ishaan-jaff in https://github.com/BerriAI/litellm/pull/6258
* Litellm router code coverage 3 by krrishdholakia in https://github.com/BerriAI/litellm/pull/6274
* Remove "ask mode" from Canary search by yujonglee in https://github.com/BerriAI/litellm/pull/6271
* LiteLLM Minor Fixes & Improvements (10/16/2024) by krrishdholakia in https://github.com/BerriAI/litellm/pull/6265


**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.49.5...v1.49.6-stable



Docker Run LiteLLM Proxy


docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.49.6-stable



Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| /chat/completions | Failed ❌ | 280.0 | 308.62173755687854 | 6.168234186408995 | 0.0 | 1846 | 0 | 209.20113499994386 | 2605.53480599998 |
| Aggregated | Failed ❌ | 280.0 | 308.62173755687854 | 6.168234186408995 | 0.0 | 1846 | 0 | 209.20113499994386 | 2605.53480599998 |