Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Token Streaming in AGS , Support Env variables #5659

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Add Token Streaming in AGS , Support Env variables #5659

wants to merge 10 commits into from

Conversation

victordibia
Copy link
Collaborator

@victordibia victordibia commented Feb 22, 2025 •
edited
Loading

Adds Token Streaming in AGS.

Agentchat now supports streaming of tokens via ModelClientStreamingChunkEvent . This PR is to track progress on supporting that in the AutoGen Studio UI.

If model_client_stream is enabled in an assitant agent, then token will be streamed in UI.

streaming_assistant = AssistantAgent(
    name="assistant",
    model_client=model_client,
    system_message="You are a helpful assistant.",
    model_client_stream=True,  # Enable streaming tokens.
)
ags_streaming_720.mov

Also adds support for env variables in AGS Settings

You can set env variables that are loaded just before a team is run. Handy to set variable to be used by tools etc.

image

Note: the set variables are available to the server process.

Why are these changes needed?

Related issue number

Closes #5627

Checks

@codecov Codecov
Copy link

codecov bot commented Feb 22, 2025 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 75.56%. Comparing base (a226966) to head (c50d0aa).

Additional details and impacted files 
@@           Coverage Diff           @@
##             main    #5659   +/-   ##
=======================================
  Coverage   75.56%   75.56%           
=======================================
  Files         171      171           
  Lines       10483    10483           
=======================================
  Hits         7921     7921           
  Misses       2562     2562
Flag Coverage Δ
unittests 75.56% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@victordibia victordibia changed the title Add Token Streaming in AGS Add Token Streaming in AGS , Support Env variables Feb 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ekzhu ekzhu ekzhu approved these changes

@husseinmozannar husseinmozannar husseinmozannar approved these changes

@jackgerrits jackgerrits Awaiting requested review from jackgerrits

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Support visualization of model streaming chunks in AutoGen Studio
3 participants
@victordibia @ekzhu @husseinmozannar