Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/concepts/stateful-agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,8 @@ No per-loop dict workarounds, no `WORKERS=1` ceremony, no surprises. The pool is

**Better escape for sync-blocking**: refactor the blocking call to `await asyncio.to_thread(blocking_call)`. The blocking call runs on Python's default thread pool; the user loop stays free; you don't need N>1 workers.

**Loops must be long-lived.** Two surfaces cache their HTTP clients per event loop, keyed on `id(loop)`: the pooled clients your injected dependencies call through, and the native Anthropic, OpenAI and Gemini LLM clients. Both assume a loop lives as long as the agent process. Do not call `asyncio.run()` or otherwise create short-lived loops inside an agent to reach them: the id of a closed loop can be reused, nothing detects that, and the call gets back a client bound to the dead loop and fails with `RuntimeError: Event loop is closed`. Run async work on the loop the runtime already gives you.

The structural answer — the one that survives replica restart, scales
horizontally, and composes with the rest of the mesh — is the
three-agent decomposition that follows.
Expand Down
2 changes: 2 additions & 0 deletions docs/python/dependency-injection.md
Original file line number Diff line number Diff line change
Expand Up @@ -362,6 +362,8 @@ own internal use of this pattern for httpx clients.
runs on Python's default thread pool; the user loop stays free; no
cross-worker resource problem.

**Loops must be long-lived.** Two surfaces cache their HTTP clients per event loop, keyed on `id(loop)`: the pooled clients your injected dependencies call through, and the native Anthropic, OpenAI and Gemini LLM clients. Both assume a loop lives as long as the agent process. Do not call `asyncio.run()` or otherwise create short-lived loops inside an agent to reach them: the id of a closed loop can be reused, nothing detects that, and the call gets back a client bound to the dead loop and fails with `RuntimeError: Event loop is closed`. Run async work on the loop the runtime already gives you.

### Trade-offs

| Concern | N=1 (default) | `MCP_MESH_TOOL_WORKERS=N` (N>1) |
Expand Down
2 changes: 2 additions & 0 deletions src/core/cli/man/content/dependency-injection.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,8 @@ app = FastMCP("my-agent", lifespan=_lifespan)

**Opt-in `MCP_MESH_TOOL_WORKERS=N` (N>1)** for tool bodies that do sync blocking work and need concurrent calls to absorb it. Loop-affinity caveat: resources created in `lifespan` bind to worker-0 only. For cross-worker access, use a per-loop dict cache (each worker lazily builds its own resource on first access). Better escape: `await asyncio.to_thread(blocking_call)` keeps the user loop free without N>1 workers.

**Loops must be long-lived.** Two surfaces cache their HTTP clients per event loop, keyed on `id(loop)`: the pooled clients your injected dependencies call through, and the native Anthropic, OpenAI and Gemini LLM clients. Both assume a loop lives as long as the agent process. Do not call `asyncio.run()` or otherwise create short-lived loops inside an agent to reach them: the id of a closed loop can be reused, nothing detects that, and the call gets back a client bound to the dead loop and fails with `RuntimeError: Event loop is closed`. Run async work on the loop the runtime already gives you.

For long-running stateful work, use MeshJob (`@mesh.tool(task=True)`) with state externalized to a separate state agent — see `meshctl man jobs` and the Stateful Agents concept doc.

For the narrow case where neither fits (sub-10ms state-mutation latency, unportable loop-bound resources like GPU contexts, true background daemons), see the In-Process State escape hatch. The default answer should still be MeshJob.
Expand Down
40 changes: 39 additions & 1 deletion src/core/cli/man/renderer_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -444,6 +444,44 @@ func TestStyleInlineNoStrayItalicBetweenCodeSpans(t *testing.T) {
// adds a span. The rewritten bullet is a list item but changes no line's markup
// shape, so the two list goldens below hold.
//
// Issue #1564: +3 / +0 / +0, one added paragraph at the end of
// `dependency-injection.md`'s "Loop topology" section. That section taught which
// loops the runtime runs your code on and never stated the lifetime the runtime
// assumes of them. The assumption was real but written only in source comments:
// the client caches are keyed on `id(loop)`, and nothing detects a closed loop
// whose id was later reused, so an agent calling `asyncio.run()` per call gets
// `RuntimeError: Event loop is closed` out of a client bound to the dead loop.
// The three spans are `id(loop)` (the cache key, which is why the reuse is
// undetectable), `asyncio.run()` as the thing not to do, and the error it
// produces. Prose, not a list item, so the two list goldens hold.
//
// The paragraph named three caches in its first draft and now names two. The
// count is coincidentally unchanged — `mesh.jobs` left and `id(loop)` arrived —
// so anyone bisecting a future delta here should not read 1790 as "the wording
// never moved". `mesh.jobs` does not belong: its cache is keyed by
// `(registry_url, job_id)` with no loop component, and the `JobProxy` inside is
// a pyo3-wrapped Rust reqwest client with no Python loop affinity at all. The
// two that do belong are `unified_mcp_proxy.py` (`_fastmcp_client_pool` and
// `_httpx_pool`, both keyed `(id(loop), endpoint)`) and the native LLM clients.
// That is THREE native clients, not the two #1564 cites: `anthropic_native.py`
// carries the same `dict[int, httpx.AsyncClient]` cache and the same ASSUMPTION
// comment as `openai_native.py` and `gemini_native.py`, so the page names
// Anthropic alongside them.
//
// The contract has THREE surfaces and one wording: this page,
// `docs/python/dependency-injection.md` and `docs/concepts/stateful-agents.md`
// each carry the byte-identical paragraph in their own "Loop topology" section.
// The `docs/python` one is the anchor `docs/concepts/in-process-state.md` and
// `stateful-agents.md` deep-link to (`#loop-topology-v224`), so omitting it
// would have left the most-linked copy of the section as the one that does not
// state the contract. Only this page is in the man corpus, so the two `docs/`
// copies move no golden here — `scripts/check_doc_claims.py` is what reads
// those, and a future edit to the wording has to touch all three files.
//
// The behaviour is Python's alone, so the `_java` and `_typescript`
// dependency-injection pages did not get it; neither has a Loop topology section
// to put it in, and the variant golden in `variant_corpus_test.go` is unmoved.
//
// Issue #1500: the sentence most annotations above end on — that the `_java`
// and `_typescript` files are invisible here, so a review of them cannot lean
// on this test — is still true of THESE constants and no longer true of this
Expand All @@ -457,7 +495,7 @@ func TestStyleInlineNoStrayItalicBetweenCodeSpans(t *testing.T) {
// the starter's POM; a test in this package cannot, and #1499 shipped three
// false claims through a green run here.
const (
wantInlineCodeSpans = 1787
wantInlineCodeSpans = 1790
wantListCodeSpans = 522
wantMarkupListLines = 448
)
Expand Down
5 changes: 0 additions & 5 deletions src/runtime/python/_mcp_mesh/engine/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@
"DependencyInjector",
"get_global_injector",
# MCP client proxies
"AsyncMCPClient",
"UnifiedMCPProxy",
"EnhancedUnifiedMCPProxy",
# Self-dependency proxy
Expand Down Expand Up @@ -54,10 +53,6 @@ def __getattr__(name):

return get_global_injector
# MCP client proxies
elif name == "AsyncMCPClient":
from .async_mcp_client import AsyncMCPClient

return AsyncMCPClient
elif name == "UnifiedMCPProxy":
from .unified_mcp_proxy import UnifiedMCPProxy

Expand Down
236 changes: 0 additions & 236 deletions src/runtime/python/_mcp_mesh/engine/async_mcp_client.py

This file was deleted.

Loading
Loading