Updating Patch Version
This patch version Introduces explicit resource management to prevent unexpected resource leaks.
By default, the library closes underlying HTTP and gRPC connections when the client is garbage-collected. However, you can now manually close the `Friendli` or `AsyncFriendli` client using the `.close()` method or utilize a context manager to ensure proper closure when exiting a `with` block.
Usage examples
python
import asyncio
from friendli import AsyncFriendli
client = AsyncFriendli(base_url="0.0.0.0:8000", use_grpc=True)
async def run():
async with client:
stream = await client.completions.create(
prompt="Explain what gRPC is. Also give me a Python code snippet of gRPC client.",
stream=True,
top_k=1,
)
async for chunk in stream:
print(chunk.text, end="", flush=True)
asyncio.run(run())