chore: mesh upgrade, clean up legacy special case code, simplify model selection for mesh - #5289
chore: mesh upgrade, clean up legacy special case code, simplify model selection for mesh#5289micspiral wants to merge 10 commits into
Conversation
Bumps the eight mesh-llm pins (six in the desktop Tauri crate, two in buzz-relay) from v0.74.0 to v0.75.0 and refreshes both lockfiles. Also deletes scripts/ensure-mesh-native-runtime.sh and its six call sites. The script built llama.cpp from source locally to pre-seed a native-runtime cache, but nothing needs that: - The app installs the signed runtime itself. `initialize_host_runtime()` reaches `install_native_runtime()`, whose `allow_download` defaults to true, and v0.75.0 publishes `native-runtimes.json` (13 artifacts, including darwin-aarch64-metal) alongside the release. Verified by fetching the manifest and artifact and checking both sha256 sidecars. - CI never called it. The mesh lifecycle workflow caches the app's own `~/.cache/mesh-llm/native-runtimes` and says so in a comment: "the mesh-llm SDK downloads a signed native runtime on first init". The desktop build job compiles llama.cpp through mesh-llm's own prepare-llama.sh / build-llama.sh. - The three `mesh-e2e-*` recipes that also used it are hand-run only, referenced nowhere else in the repo. `mesh=1` behaviour is unchanged; only the pre-seed line is gone from dev / staging / production. KNOWN ISSUE (do not merge before resolving): on a machine that already has a pre-0.75 runtime cache, startup fails with "native runtime artifact ... does not declare file checksums". v0.75.0 added `NativeRuntimeManifest::verify_contents`, which requires per-file checksums; caches written by older loaders have none, and the startup path enumerates the whole cache strictly, so one stale entry aborts the scan even when a valid 0.75.0 entry is present. Reproduced and bisected here: failing with the stale entries present, passing with only 0.75.0 cached, and passing with the stale entries present once a bundle dir is supplied. That is why the installed CLI is unaffected -- it ships a runtime directory beside its binary, which is discovered leniently. Signed-off-by: Michael Neale <michael.neale@gmail.com>
The runbook told readers to run scripts/ensure-mesh-native-runtime.sh, which this branch removes. The desktop installs the signed runtime itself on first init, so the only manual step left is clearing stale pre-0.75 cache entries. Signed-off-by: Michael Neale <michael.neale@gmail.com>
…ttee Shared-compute `auto` now maps straight onto MeshLLM's virtual `mesh` model and sends it unconditionally. MeshLLM >= 0.75.0 resolves that per request: a Mixture-of-Agents committee when two or more workers are reachable, and a single served model when they are not (`moa_gateway::degrade_to_single_model`). Before 0.75.0 a `model=mesh` request could 503 on a lone node, so the agent kept its own hysteretic view of whether a committee was currently possible and only then dared send `mesh`. That client-side machinery is now redundant, and removing it deletes the polling and the debounce it needed to avoid flapping: - `resolve_openai_model`, `observe_mesh_virtual_model`, `cool_down_collective`, `mesh_catalog_supports_collective` - `MeshAutoState`, `MeshCatalogObservation`, and the `Llm` field holding it - the `/models` probe: 5s catalog TTL, 2s probe timeout, 30s cooldown, and the two-observation confirmation count - `PostError::MeshFallback` with `is_mesh_moa_unavailable_body` / `is_mesh_moa_failure_body` and the `detect_mesh_fallback` parameter - `looks_like_unstructured_tool_call`, whose only call site was gated on the same adaptive-mesh flag `looks_like_unstructured_tool_call` arrived with this feature (#2825) rather than as a general guard, so it leaves with it. It retried once through `auto` when a committee answered tool-call markup as prose; if that turns out to matter in its own right it should come back provider-agnostic, not gated on shared compute. Behaviour that is deliberately unchanged: an explicit model is still sent verbatim and never rewritten, and plain OpenAI `auto` (a real provider model name) is still left alone. What does change is that a 503 from an explicit `mesh` request now takes the ordinary transport retry under the same model instead of failing fast to a second model -- there is no second model to fall back to once MeshLLM resolves `mesh` itself. Three test helpers (`moa_failure`, `model_catalog`, `complete_model_with_tool`) went unused once the fourteen catalog/debounce tests were removed. New tests pin the surviving contract: shared-compute `auto` sends `mesh` and never requests `/v1/models`, plain `auto` is untouched, and an explicit model keeps its name across retries. Signed-off-by: Michael Neale <michael.neale@gmail.com>
v0.75.1 carries the upstream fix (Mesh-LLM/mesh-llm#1196): startup now scans the native-runtime cache leniently (`installed_lenient`), skipping unusable entries instead of aborting the whole scan on the first one. That removes the blocker this branch has carried since the v0.75.0 bump. On a machine with a pre-0.75 cache, v0.75.0 failed with: native runtime artifact meshllm-native-runtime-darwin-aarch64-metal does not declare file checksums because caches written by older loaders have no per-file checksums, and the strict enumeration covered every version under the cache root. Verified on a machine that still has the offending entries in place (0.73.1, 0.74.0, 0.74.0-rc8 all with `runtime.files = 0`), with no bundle directory and no MESH_LLM_BUILD_VERSION override: - no checksum abort - the v0.75.1 runtime installed itself alongside the stale entries - crates/buzz-relay/examples/mesh_agent_e2e passed 4/4 (explicit-model chat, auto-model chat, oversized-budget rejection, agentic tool use) Pins move in both manifests (six mesh-llm crates in the desktop Tauri crate, two in buzz-relay) with both lockfiles refreshed. The v0.75.1 release publishes native-runtimes.json with 13 artifacts including darwin-aarch64-metal at Skippy ABI 0.1.35, which matches what the crates compile against. Signed-off-by: Michael Neale <michael.neale@gmail.com>
Shared compute either uses MeshLLM's virtual `mesh` model or a model the user
named. Nothing else. That decision now happens once, in the desktop
translation layer that already turns the native provider into an
OpenAI-compatible transport:
stored "auto" (or blank legacy) -> BUZZ_AGENT_MODEL=mesh
stored "<a model>" -> BUZZ_AGENT_MODEL=<a model>
MeshLLM resolves `mesh` per request -- a Mixture-of-Agents committee when two
or more workers are reachable, a single served model when they are not
(`moa_gateway::degrade_to_single_model`) -- so the caller never has to know
which of those will happen.
buzz-agent therefore needs no mesh concepts at all. Removed:
- the `auto` -> `mesh` mapping in `openai_request`
- `MESH_VIRTUAL_MODEL_ID` / `MESH_AUTO_MODEL_ID`
- `Config::prefer_mesh_for_auto` and the `BUZZ_AGENT_PREFER_MESH_FOR_AUTO`
env var that fed it
`openai_request` now sends the configured model verbatim, which is what it
does for every other provider. `auto` stays a real provider model name for
plain OpenAI hosts, unchanged, because nothing rewrites it any more.
The previous commit moved this decision from a `/models` probe into a
compile-time mapping; this moves it out of the agent entirely. The stored
value and the picker label both stay "auto" -- that is the user's word for
"let the mesh decide", and it is still what gets persisted.
Tests: `relay_mesh.rs` gains three that pin the translation (stored `auto`
and blank both become `mesh`; a named model is sent verbatim). In
`llm.rs`, the two mapping tests are replaced by one asserting the configured
model reaches the wire unchanged for `auto`, `mesh`, and a named model, with
no catalog request on any of them.
Signed-off-by: Michael Neale <michael.neale@gmail.com>
Two env vars told the agent process which model to use, and only one of them was translated. `BUZZ_AGENT_MODEL` said `mesh`; `BUZZ_ACP_MODEL` was written straight from the stored value and still said `auto` (`runtime.rs`, before the mesh override runs). `auto` is not a model the mesh advertises, so buzz-acp rejected it as unavailable and fell back to its default with a warning on every new session — recoverable, but the two consumers were being told different things. `relay_mesh_wire_model` is now the one place a stored value becomes a wire name: `auto` (and a blank legacy value) becomes MeshLLM's virtual `mesh` model, anything else is a model the user named and passes through unchanged. Both the LLM transport env and the ACP harness go through it, so they cannot disagree. Also narrows a comment I had overstated. Sending `mesh` is not "always safe": `degrade_to_single_model` is a pre-flight capacity decision, so a committee that forms and then loses a worker still surfaces as a failed turn. MoA repairs partial results internally (`repair_tool_result_answer`, `fallback_arguments`) before it reaches that point, which is why no client-side reroll is being added back. The headless harness now exercises the shipped path. P2, P3, and P4 send `mesh` where they previously sent `auto` (MeshLLM's own router) or the physical model id (bypassing mesh routing entirely) — so no leg was covering what Buzz actually puts on the wire. With one served model there is no committee, which makes these a direct test of the degradation path. Verified on this machine at this commit, three models, 4/4 each: gemma-4-E4B-it-Q4_K_M, gemma-4-26B-A4B-it-UD-Q4_K_M, Qwen3-8B-Q4_K_M. P4 is the load-bearing one: a real ACP tool call through `mesh` into buzz-dev-mcp, asserted by reading `BUZZ_OK` back off disk rather than trusting the agent's own account of itself. P3 confirms the wire name in its error text (`llm: (mesh) exhausted retries: 503`). The harness comment claiming it supplies "exactly" what `apply_relay_mesh_env` does was wrong — it omits `BUZZ_AGENT_REQUIRE_REPLY`, which needs Buzz's publish tools to mean anything. It now says transport subset. Signed-off-by: Michael Neale <michael.neale@gmail.com>
origin/main added `post()` call sites carrying `detect_mesh_fallback`, which this branch removed along with `PostError::MeshFallback`. Neither side touched the same lines, so the merge applied cleanly and did not compile — the failure only appears in `--all-targets`, which is why per-crate `cargo check` missed it and CI (testing the merge result) did not. Signed-off-by: Michael Neale <michael.neale@gmail.com>
`wait_for_mesh_inference` posted the caller's stored model id. For shared-compute `auto` that is not a name the mesh advertises, so readiness validated a route no agent uses — and could fail startup while the real route worked. It now translates through `relay_mesh_wire_model`, the same mapping the spawn env and the ACP harness use. Named models pass through unchanged, so the serve-side callers that already pass a real model are unaffected. Translating inside the probe rather than at its seven call sites keeps the mapping in one place, which was the point of the previous commit. Also corrects the e2e runbook: it still told readers to clear stale pre-0.75 runtime cache entries by hand, which is exactly what pinning v0.75.1 removed the need for. Signed-off-by: Michael Neale <michael.neale@gmail.com>
…n retired #5404 allowed RUSTSEC-2026-0243 with the removal condition written into the comment: "Remove after mesh-llm migrates to nostr-sdk >= 0.45, which absorbed the standalone relay pool." MeshLLM v0.75.1 does exactly that, so nostr-relay-pool is no longer in either lockfile and the advisory has nothing to fire on. That PR measured 13 MeshLLM API compilation errors from bumping nostr-sdk to 0.45 against v0.74; upstream did the migration in v0.75, which is why this needs no source change here. Verified with the same commands #5404 used: bin/cargo-deny --locked check --config deny.toml advisories, and bin/cargo-deny --locked check -- advisories ok, bans ok, licenses ok, sources ok. Leaving a satisfied exception in place would suppress a real future advisory for this crate, so it goes with the bump that retired it rather than waiting for someone to notice. Signed-off-by: Michael Neale <michael.neale@gmail.com>
wesbillman
left a comment
There was a problem hiding this comment.
Reviewed on Wes's behalf at 8e3b15019d09a17de0cd21144ac62b366f7ac5d7.
No blocking findings.
I traced the effective provider/model resolution through EffectiveAgentConfig::relay_mesh_model_id, spawn_agent_child, the mesh process env, BUZZ_ACP_MODEL, and wait_for_mesh_inference. Stored auto/legacy blank consistently becomes the wire model mesh; explicit model names stay explicit. Linked definitions and global inheritance continue to use the existing effective-config precedence rather than stale instance bytes.
I also checked the removed buzz-agent catalog/hysteresis/fallback path, all six deleted Justfile pre-seed call sites, both dependency manifests/lockfiles, the retired advisory exception, and the focused e2e changes. The removed fallback's behavioral change is accurately disclosed: terminal committee failure retries the same mesh route rather than changing models. MeshLLM 0.75's single-worker degradation makes the old eligibility fallback obsolete, while an already-formed committee losing a worker remains a known two-worker failure case.
Validation evidence: git diff --check passed locally on the exact head; all GitHub checks are green, including Unit Tests, Rust Lint, Desktop Core, Security, Relay E2E, and Relay-Driven Mesh Lifecycle Smoke. I did not duplicate the broad CI suite locally.
One non-blocking caveat: BUZZ_ACP_MODEL remains intentionally user-overridable through layered env, so “cannot disagree” applies to Buzz's derived defaults, not an explicit power-user override.
|
🤖 Reviewed at head 8e3b150 — no blocking concerns, but I think the The branch at line ~632 still compares The happy path is fine — the chat probe sends Happy to be corrected if the single-worker catalog behaves differently in practice. |
Shared compute now has exactly two model choices: MeshLLM's virtual
meshmodel, or a model you name. Buzz picks between them in one place, and
buzz-agent no longer knows meshes exist.
What changed
degrade_to_single_model, so amodel=meshrequest is answered by one served model when there is nocommittee to form, instead of failing. v0.75.1 adds Mesh-LLM#1196, which
skips stale pre-0.75 runtime cache entries rather than aborting startup on
them — without it, anyone who had run mesh on 0.73/0.74 could not start.
/v1/models(5s TTL, 30s cooldown, two-observation debounce) to decidewhether
meshwas safe to send. MeshLLM now decides per request, so thepolling, its hysteresis, and its 503 fallback are gone.
relay_mesh_wire_model()turns the stored value intoa wire name:
autobecomesmesh, a named model passes through. The spawnenv, the ACP harness, and the readiness probe all use it, so they cannot
disagree — previously
BUZZ_ACP_MODELand the probe both saidauto, a namethe mesh does not advertise.
nostr-relay-pooladvisory exception. ci(security): allow retired relay pool advisory #5404 allowedRUSTSEC-2026-0243 "after mesh-llm migrates to nostr-sdk >= 0.45". v0.75.1
does, so the retired crate is gone from both lockfiles and the exception
would only mask a future advisory for it.
scripts/ensure-mesh-native-runtime.shand its six justfile callsites. It built llama.cpp from source into the runtime cache; the app already
downloads the signed release runtime itself, and CI never called it.
Why it is better
−639 lines of Rust. Availability is decided by the node that knows the
answer, per request, instead of by a client cache that could be stale for up to
30 seconds. A second worker joining now takes effect on the next request rather
than after two confirming probes.
Behaviour change
A 503 on an explicit
meshrequest takes the ordinary transport retry underthe same model instead of failing over to a second one — there is no second
model to fail over to now. MoA repairs partial committee results internally
before it reaches that point.
Validation
crates/buzz-relay/examples/mesh_agent_e2e.rsnow sendsmeshwhere itpreviously sent
autoor the physical model id, so no leg was covering whatBuzz actually puts on the wire. 4/4 on gemma-4-E4B, gemma-4-26B-A4B, and
Qwen3-8B — including a real ACP tool call through
meshinto buzz-dev-mcp,asserted by reading the written file back off disk.
Hand-tested in the desktop app on both gemma-4 sizes: picked Auto, agent logged
model_id=mesh, replied in channel.Not covered
A committee that forms and then loses a worker returns 502, and that needs two
workers to reproduce — not testable on one machine.