Skip to content

Support streaming responses for accurate token counting #2

@manja316

Description

@manja316

When using streaming (stream=True), token counts are not available until the stream completes. Currently guard.record() requires known token counts upfront.

Need a context manager or callback that accumulates tokens from stream chunks:

with guard.stream_tracker(model="openai/gpt-4o") as tracker:
    for chunk in stream:
        tracker.add_chunk(chunk)
    # auto-records on exit

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions