Big update!
For the longest time, openai has not allowed us to count tokens when streaming. This is incredibly annoying, as most calls made on production sites will be streamed. This update finally does the calculation for you, accounting for all the different edge cases (images, different models, additional tokens from hidden tags), and allows you to access the token count easily from the .usage object.
This update has also removed the retry() and timeout() wrappers, as I felt that most people would not use it, and if you did, it would probably be better if you wrote your own custom solution with your own logging. This update also changes the object types of the usage parameters, so double check that this will not break anything.
How to count usage when streaming!
py
from flowchat import Chain
from PIL import Image
naruto_image = Image.open("examples/images/naruto.png")
character_chain = (
Chain(model="gpt-4-turbo")
.link("What is this image?", images={"url": naruto_image, "format_type": "PNG"})
)
for token in character_chain.stream(plain_text_stream=True):
print(token)
Direct access:
usage = character_chain.usage -> {'prompt_tokens': 777, 'completion_tokens': 68}
character_chain.log_tokens()