Repository navigation
Conversation
deepcopy in two hot paths crashed on non-copyable objects (locks, DB connections, sockets) and silently cloned stateful objects so writes never reached the caller. - connection.py: dict(flow._context) gives per-run isolation without deep-copying user values - flow.py update_context: copy.copy(self) + shallow dict merge avoids deep-copying the whole Flow including callbacks - model_invoker.py create_with_llm_observe: list(middleware) passes instances through by reference, matching MiddlewareChain behavior, so stateful middleware (rate limiters, counters) accumulates state correctly across LLM calls Fixes #1675 Signed-off-by: Christian-Sidak <61099993+Christian-Sidak@users.noreply.github.com>
…ions - Replace the closure-based model middleware tests, which passed without the fix, with ones that keep state on the middleware instance and build one invoker per agent invocation. - Add end-to-end MaxCalls tests for the model_middleware slot, shared across invocations and across agents, plus a prebuilt Lock smoke test. - Cover the Flow context contract both ways: values shared by reference (Lock, sqlite3 connection, nested dict writes) and top-level put/delete isolated per run, including concurrent runs and update_context. - Update the Flow docs and the ModelInvoker docstring to match. Signed-off-by: Guan Zheng Huang <guanzheng.huang@hotmail.com>
… copying it in couple Middleware instances are shared policy objects, so tracking the creation event with a per-instance flag meant a shared middleware only announced itself in the first run of the process. Session documents resolve middleware names per session, so from the second run on a guardrail trace lost its rail_name. The registry now lives on the session and is shared by every frame derived from it. Node.extend_middleware and extend_model_middleware (used by rt.couple) no longer deep-copy the existing middleware, so a coupled node shares the original's MaxCalls budget, Lock or ConversationMemory, and middleware holding a non-copyable resource no longer raises. ConversationMemory's __deepcopy__ only existed to work around that copy and is removed. Fixes #1675
8 tasks done
CoronRing
added a commit
that referenced
this pull request
Oct 29, 2026
…py (#1695) ## Summary - connection.py: replace deepcopy(flow._context) with dict(flow._context) to give per-run isolation without deep-copying user-supplied values - flow.py update_context: replace deepcopy(self) with copy.copy(self) and a shallow dict merge, avoiding a full deep-copy of the Flow including callbacks on every call - model_invoker.py create_with_llm_observe: replace deepcopy(middleware) with list(middleware), passing instances through by reference so stateful middleware accumulates state correctly across LLM calls The dict(...) shallow copy preserves the existing contract: rt.context.put(...) inside a run writes to a per-run copy of the top-level dict, so it cannot leak into flow._context or the next run. ## Test plan - [x] test_flow_invoke_passes_non_copyable_context_object: threading.Lock in context no longer raises TypeError - [x] test_flow_invoke_context_value_is_same_object_not_clone: run sees the callers exact object (is check) - [x] test_flow_update_context_non_copyable_object: update_context with a lock in context no longer raises - [x] test_flow_update_context_does_not_modify_original: original Flow is unchanged after update_context - [x] test_flow_context_mutations_isolated_between_runs: rt.context.put inside a run does not bleed into flow._context - [x] test_create_with_llm_observe_preserves_stateful_middleware: counting middleware accumulates state across two invoke calls (reads 2, not 0) - [x] test_create_with_llm_observe_non_copyable_middleware_does_not_crash: middleware holding a threading.Lock no longer raises TypeError - [x] Existing test suite passes: all 30 unit tests and 21 integration flow tests green locally Part of #1675 (finished in #1758) --------- Signed-off-by: Christian-Sidak <61099993+Christian-Sidak@users.noreply.github.com> Signed-off-by: Guan Zheng Huang <guanzheng.huang@hotmail.com> Co-authored-by: Guan Zheng Huang <guanzheng.huang@hotmail.com> Co-authored-by: Guan Huang <89324736+CoronRing@users.noreply.github.com>
Pooria90
reviewed
Oct 31, 2026
Pooria90
left a comment
Contributor
There was a problem hiding this comment.
Thanks @CoronRing.
The PR looks clean to me and nothing is blocking. I have two comments:
- Please edit the comments that refer to the old process-wide middleware. My Claude flagged these:
utils/logging/run_view.py:62cli/viz_api/queries/_common.py:208-212cli/viz_api/queries/middleware.py:139-146cli/viz_api/queries/guardrails.py:46-49
- The fact that middleware are shared now, make the concurrent sessions fail esp if the user does not properly handle stateful middleware. We need to document this behavior, refer them to proper implementations like
MaxCallsand likely provide some tools for them so they can make stateful middleware more easily.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Finishes #1675 on top of #1695. Stacked on #1695: merge that first. Until then this diff also shows its commits; the change here is only the last commit,
fix(middleware): announce shared middleware once per session ....#1695 stops deep-copying model middleware, so one instance is now shared across every run. That exposed two things:
Middlewareemitted itsmiddleware.creationevent once per instance (a_has_registeredflag). Session documents resolve middleware names from that session's creation events only (observability/node_internals.py), so a shared guard'srail_namecame out as""in every run after the first. Node-level middleware already had this problem onmain; fix: shallow-copy flow context and model_middleware instead of deepcopy #1695 extends it tomodel_middleware, where guardrails live.rt.couplestill deep-copied the existing middleware (Node.extend_middleware/extend_model_middleware). A coupled agent got its ownMaxCallsbudget and its ownLock, and middleware holding athreading.Lockor DB connection raisedTypeError.Changes:
SessionContextand is passed by reference to every frame derived from it (with_scope_pushed,restore_scope), so it lives exactly as long as the session.claim_middleware_registrationincontext/central.pyreturnsFalseoutside a session, whereemithas nowhere to publish, so a call outside a run no longer uses up the first run's announcement.rt.coupleshares instances.extend_middleware/extend_model_middlewarebuild a new list but keep the instances.ConversationMemory.__deepcopy__. It only existed to work around the copy inextend_middleware.coupledocstring and theMaxCallssharing note.Fixes #1675
Type of change
Checklist
ruff check . && ruff format .)pytest tests)Notes
Tests. All 17 new tests fail with this commit's source changes reverted and pass with them:
restore_scopeshare it; once per session; new session resets; nothing claimed outside a sessionrail_nameis set in both of two runs of the same flow (integration)rt.couplefunction_node); middleware holding athreading.Lockorsqlite3.Connection; coupled agent spends the originalMaxCallsbudgetConversationMemoryFull unit + integration suite: 4076 passed. The only failures were the 2 MCP integration tests, which
pip installan MCP server and couldn't in my local env.mkdocs build --strictpasses.Behaviour change. A coupled node now shares its middleware with the node it came from, as
agent_node/function_nodealready do (see the comment innode_builder.py). ForMaxCallsandLockthat's the point. For a custom stateful middleware it means state is shared across the original and coupled node rather than starting fresh.Not changed.
Node.safe_copystill deep-copies node attributes. I checked it with a non-copyable object passed as a node argument and the run,to_vertices()andgraph_serialization()all work, since node instances don't keep their call arguments. So I left it out of scope.