You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix(middleware): fill context placeholders once per model call - #1753
ContextInjection filled placeholders in the agent's own message history, in place, before every model call. Once a tool call sent the conversation back to the model, the already-filled messages were filled a second time. So text passed through rt.escape_braces was only delivered as written on the first call, and a context value containing {other_key} was expanded into that key's value on the next call.
Each message used to be marked as filled after the first pass. #1441 removed that marker along with the inject_prompt switch, so this regressed in 1.5.0.
ContextInjection now fills a copy of the history and passes that copy on, so the caller's messages are never changed.
Added a regression test: an agent with one tool, two model calls, and both cases above.
Type of change
Bug fix
Feature
Breaking change
Docs
Refactor / chore / build / tests
Checklist
Lint & format pass (ruff check . && ruff format .)
Tests added/updated and pass locally (pytest tests)
Docs updated if user-facing behavior changed
Breaking changes include migration notes
Notes
The conversation an agent returns now keeps the placeholders as written (for example {user_name}) rather than the filled-in values. The model and the recorded LLM events still see the filled prompt.
Follow-up: _wire_history in llm_helpers.py still copies the system message and gives in-place filling as the reason; that reason no longer holds.
deepcopy(message_history) runs on every model call and copies everything in the history, including attachments (base64 images, PDFs) and tool results, even though only UserMessage and SystemMessage string content is ever filled. fill_prompt reassigns _content rather than mutating it, so a shallow copy of each message is enough:
I swapped this in, and your new test plus the prompt, middleware and built_nodes suites all pass. It also lines up with #1695, which is moving the hot paths off deepcopy because it is slow on large values and crashes on non-copyable ones. _llm_observe already deep-copies the history once per call, so this isn't a new failure mode, just a second full copy per round trip that we don't need.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ContextInjectionfilled placeholders in the agent's own message history, in place, before every model call. Once a tool call sent the conversation back to the model, the already-filled messages were filled a second time. So text passed throughrt.escape_braceswas only delivered as written on the first call, and a context value containing{other_key}was expanded into that key's value on the next call.Each message used to be marked as filled after the first pass. #1441 removed that marker along with the
inject_promptswitch, so this regressed in 1.5.0.ContextInjectionnow fills a copy of the history and passes that copy on, so the caller's messages are never changed.Type of change
Checklist
ruff check . && ruff format .)pytest tests)Notes
The conversation an agent returns now keeps the placeholders as written (for example
{user_name}) rather than the filled-in values. The model and the recorded LLM events still see the filled prompt.Follow-up:
_wire_historyinllm_helpers.pystill copies the system message and gives in-place filling as the reason; that reason no longer holds.