Usage counting when streaming!

Big update!

For the longest time, openai has not allowed us to count tokens when streaming. This is incredibly annoying, as most calls made on production sites will be streamed. This update finally does the calculation for you, accounting for all the different edge cases (images, different models, additional tokens from hidden tags), and allows you to access the token count easily from the .usage object.

This update has also removed the retry() and timeout() wrappers, as I felt that most people would not use it, and if you did, it would probably be better if you wrote your own custom solution with your own logging. This update also changes the object types of the usage parameters, so double check that this will not break anything.

How to count usage when streaming!

from flowchat import Chain
from PIL import Image

naruto_image = Image.open("examples/images/naruto.png")

character_chain = (
    Chain(model="gpt-4-turbo")
    .link("What is this image?", images={"url": naruto_image, "format_type": "PNG"})
)

for token in character_chain.stream(plain_text_stream=True):
    print(token)

# Direct access:
# usage = character_chain.usage # -> {'prompt_tokens': 777, 'completion_tokens': 68}
character_chain.log_tokens()

v1.1.0

Small updates, specifically with json_schema, stopping infinite retries, and TYPES! Flowchat is now compatible with pylance on strict mode :)

v1.0.0

The first production ready release of flowchat, also here on github!

• Includes streaming! Accessible through the .stream() function, you can use this instead of a final pull; as it directly returns a generator, you should only use this function at the end. Due to openai complications, token count will not be counted with streams.
• Check out example usage in examples!

I'm also happy to say that flowchat has worked very well in many of my personal projects as well; we have some plans for taking it to the next level, hopefully it'll get released! Let me know if you have any feedback as well.

Cheers!