Make serverless hooks work better - #84
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
anbuzin
reviewed
May 8, 2026
| self, | ||
| gen: AsyncGenerator[types.events.Event], | ||
| *, | ||
| message: types.messages.Message | None = None, |
Collaborator
There was a problem hiding this comment.
Should probably call it seed_message or something explicit like that
anbuzin
reviewed
May 8, 2026
| If the last message is an assistant turn with tool calls, replay | ||
| that turn as synthetic stream events instead of calling the model | ||
| — useful for resume-after-approval flows where the assistant turn | ||
| is already in history and re-asking would give a different answer. |
Collaborator
There was a problem hiding this comment.
I wonder if we can come up with a more explicit way to signal the intent to replay. Something like:
async with ai.replay(ai.stream(**args)) as stream:
passwdyt?
Contributor
Author
There was a problem hiding this comment.
Depends if the goal is to make it work without changes to the event loop
* Add a notion of an event being marked as `replay`, which can be fed to the agent loop to trigger tool calls, but won't get returned to the user. * When the last message when calling `run` is an assistant reply with tool results, we treat that as a runnable state and replay the ToolEnds into the agent loop * ai-sdk inbound strips internal hook messages and returns the list of approvals granted by hooks, instead of the existing behavior of it signalling the hooks itself. For follow-up: the serverless-style loops are much nicer now, though still don't match the stock loop's ToolRunner pattern. The current issue, which I am going to think about some more, is that in the interleaved streaming flow, if we immediately bail out of the loop when a hook is requested, then that prevents a message from getting emitted at all.
… level decisions based on the replay flag
msullivan
force-pushed
the
hook-investigate
branch
from
May 9, 2026 17:58
cd3f8ab to
689276c
Compare
anbuzin
approved these changes
May 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
replay, which can befed to the agent loop to trigger tool calls, but won't get
returned to the user.
runis an assistant reply withtool results, we treat that as a runnable state and replay the ToolEnds
into the agent loop
of approvals granted by hooks, instead of the existing behavior
of it signalling the hooks itself.
For follow-up: the serverless-style loops are much nicer now, though
still don't match the stock loop's ToolRunner pattern. The current
issue, which I am going to think about some more, is that in the
interleaved streaming flow, if we immediately bail out of the loop
when a hook is requested, then that prevents a message from getting
emitted at all.