Litellm

What's Changed
1.[Feat] litellm.acompletion() make Langfuse success handler non blocking by ishaan-jaff in https://github.com/BerriAI/litellm/pull/1519
- The Langfuse Success Callback was blocking running `litellm.acompletion()` calls. fixed on this release
- Support for logging Cache Hits on Langfuse:
support for tagging `cache_hits` on Langfuse
(note you need langfuse>=2.6.3

<img width="1054" alt="Screenshot 2024-01-19 at 11 36 47 AM" src="https://github.com/BerriAI/litellm/assets/29436595/d65ec033-390e-4a06-b549-74625bcde6c0">

2. Langsmith: Add envs for project/run names; fix bug with None metadata by timothyasp in https://github.com/BerriAI/litellm/pull/1524
[Feat] Router improvements by ishaan-jaff in https://github.com/BerriAI/litellm/pull/1525

3. Allow overriding headers for anthropic by keeganmccallum in https://github.com/BerriAI/litellm/pull/1513
* fix(utils.py): add metadata to logging obj on setup, if exists (fixes max parallel request bug) by krrishdholakia in https://github.com/BerriAI/litellm/pull/1531
4. test(tests/): add unit testing for proxy server endpoints by krrishdholakia in https://github.com/BerriAI/litellm/commit/f5ced089d6f0af05600062e25a981fdabebba815
* fix(proxy_server.py): users can now see their key info by krrishdholakia in https://github.com/BerriAI/litellm/commit/f5ced089d6f0af05600062e25a981fdabebba815
* fix(proxy_server.py): model info now restricts for user access by krrishdholakia in https://github.com/BerriAI/litellm/commit/f5ced089d6f0af05600062e25a981fdabebba815

New Contributors
* timothyasp made their first contribution in https://github.com/BerriAI/litellm/pull/1524
* keeganmccallum made their first contribution in https://github.com/BerriAI/litellm/pull/1513

**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.18.5...v1.18.6

What's Changed
* add headers to budget manager by HaruHunab1320 in https://github.com/BerriAI/litellm/pull/1506
* Adds s3_path prefix so that we can save objects in predifined location in s3 bucket by duarteocarmo in https://github.com/BerriAI/litellm/pull/1499
* nit: switch to valid SPDX license identifier `MIT` in pyproject.toml by ErikBjare in https://github.com/BerriAI/litellm/pull/1515

New Contributors
* HaruHunab1320 made their first contribution in https://github.com/BerriAI/litellm/pull/1506

**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.18.4...v1.18.5

What's Changed
[Feat] Proxy - Add Spend tracking logs by ishaan-jaff in https://github.com/BerriAI/litellm/pull/1498
New SpendTable when Using LiteLLM Virtual Keys - Logs API Key, CreatedAt Date + Time, Model, Spend, Messages, Response
Docs to get started: https://docs.litellm.ai/docs/proxy/virtual_keys


[Feat] Proxy - Track Cost Per User (Using `user` passed to requests) by ishaan-jaff in https://github.com/BerriAI/litellm/pull/1509

- Proxy Server Track Cost Per User
Request:
shell
curl --location 'http://0.0.0.0:8000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer sk-RwPq' \
--data ' {
"model": "BEDROCK_GROUP",
"user": "litellm-is-awesome-user",
"messages": [
{
"role": "user",
"content": "what llm are you-444"
}
],
}'



Cost Tracked in LiteLLM Spend Tracking DB

<img width="419" alt="Screenshot 2024-01-18 at 5 56 17 PM" src="https://github.com/BerriAI/litellm/assets/29436595/700732e1-868a-4cec-bd17-376d7d510bab">

Notes:
- If a `user` is passed to the request the proxy tracks cost for it
- If the `user` does not exist in the User Table, we make a new user with the spend

feat(parallel_request_limiter.py): add support for tpm/rpm rate limits for keys by krrishdholakia in https://github.com/BerriAI/litellm/pull/1501



**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.18.3...v1.18.4

What's Changed

* [Feat] /key/generate - create keys with`team_id` by ishaan-jaff in https://github.com/BerriAI/litellm/pull/1500
Set `team_id` when creating users, keys
docs: https://docs.litellm.ai/docs/proxy/virtual_keys#request
shell
curl 'http://0.0.0.0:8000/key/generate' \
--header 'Authorization: Bearer <your-master-key>' \
--header 'Content-Type: application/json' \
--data-raw '{
"models": ["gpt-3.5-turbo", "gpt-4", "claude-2"],
"team_id": "core-infra"
}'


* (fix) read azure ad token from optional params extra body by krrishdholakia in https://github.com/BerriAI/litellm/commit/e0aaa94f28304e362dabd39b164940834ac0fa50

**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.18.2...v1.18.3

What's Changed
* [Test+Fix] /Key/Info, /Key/Update - Litellm unit test key endpoints by ishaan-jaff in https://github.com/BerriAI/litellm/pull/1496
* fix(ollama_chat.py): use tiktoken as backup for prompt token counting by puffo in https://github.com/BerriAI/litellm/pull/1495
* fix(parallel_request_limiter.py): decrement count for failed llm calls by krrishdholakia in https://github.com/BerriAI/litellm/commit/1ea3833ef7dad9d8fb6be32d724a5d81e496d358
* fix(proxy_server.py): show all models user has access to in /models by krrishdholakia in https://github.com/BerriAI/litellm/commit/c8dd36db9ec359fa174dcecdd6ed3c4ccf1d40de

New Contributors
* puffo made their first contribution in https://github.com/BerriAI/litellm/pull/1495

**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.18.1...v1.18.2

What's Changed
* Altered requirements.txt to require pyyaml 6.0.1 which will resolve 1488 by ShaunMaher in https://github.com/BerriAI/litellm/pull/1489
* Update s3 cache to support folder by duarteocarmo in https://github.com/BerriAI/litellm/pull/1494

New Contributors
* ShaunMaher made their first contribution in https://github.com/BerriAI/litellm/pull/1489
* duarteocarmo made their first contribution in https://github.com/BerriAI/litellm/pull/1494

**Full Changelog**: https://github.com/BerriAI/litellm/compare/v1.18.0...v1.18.1