What's Changed 🐬
1. [FEAT] Azure Pricing - based on base_model in model_info
2. [Feat] Semantic Caching - Track Cost of using embedding, Use Langfuse Trace ID
3. [Feat] Slack Alert when budget tracking fails
1. [FEAT] Azure Pricing - based on base_model in model_info by ishaan-jaff in https://github.com/BerriAI/litellm/pull/1874
Azure Pricing - Use Base model for cost calculation
Why ?
Azure returns `gpt-4` in the response when `azure/gpt-4-1106-preview` is used, We were using `gpt-4` when calculating response_cost
How to use - set `base_model` on config.yaml
yaml
model_list:
- model_name: azure-gpt-3.5
litellm_params:
model: azure/chatgpt-v-2
api_base: os.environ/AZURE_API_BASE
api_key: os.environ/AZURE_API_KEY
api_version: "2023-07-01-preview"
model_info:
base_model: azure/gpt-4-1106-preview
View Cost calculated on Langfuse
This used the correct pricing for `azure/gpt-4-1106-preview` = `(9*0.00001) + (28*0.00003)`
<img width="938" alt="Screenshot 2024-02-07 at 4 39 12 PM" src="https://github.com/BerriAI/litellm/assets/29436595/9edd3b8f-15d3-4c7f-82f7-2a0e3c08c17d">
2. [Feat] Semantic Caching - Track Cost of using embedding, Use Langfuse Trace ID by ishaan-jaff in https://github.com/BerriAI/litellm/pull/1878
- If a `trace_id` is passed we'll place the semantic cache embedding call in the same trace
- We now track cost for the API key that will make the embedding call for semantic caching
<img width="1002" alt="Screenshot 2024-02-07 at 7 18 57 PM" src="https://github.com/BerriAI/litellm/assets/29436595/203e2d12-9d1e-4411-a1dd-4219de83a2b7">
3. [Feat] Slack Alert when budget tracking fails by ishaan-jaff in https://github.com/BerriAI/litellm/pull/1877
<img width="913" alt="Screenshot 2024-02-07 at 8 08 27 PM" src="https://github.com/BerriAI/litellm/assets/29436595/4c70c204-05bf-412d-8efe-18e25a8b8b17">
**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.23.1...v1.23.2