Provider onboarding: a provider is told it cannot run tasks before a member's task dies on it - #56
Merged
Merged
Conversation
…pwatch ND-19. `test_tool_activity_is_published_while_the_agent_is_still_running` asserted the tool event arrived within 1.5s against a child that slept 3s. It measured 2.90s under the very load this feature creates — the machine at 9.03 with 29 `claude` processes of its own test round — and read as a regression when the property it names still held: 2.90s was still inside the child's sleep. The threshold was wrong, not the publisher. Any absolute bound there measures the machine as much as the code. The child now blocks on a gate this test holds shut, so it cannot reach its result line — let alone exit — while the gate is closed. An event that arrives in that window arrived while the agent was still running, on any machine at any load. Both controls are machine-checked: a publisher that buffered to the end delivers nothing until the child exits and the wait runs out; and `task.result` must be ABSENT from the same snapshot, which is what proves the child was still running rather than already finished — something an arrival time can never establish. Mutation, to prove the new shape still catches the defect the old one did: hold every line in the pump until both pipes hit EOF, then replay. The task still reports `completed` with the right output, and the test fails on the intended assertion — "no tool activity was published while the child was blocked". The gate is opened in a `finally`, so a failed assertion costs a second rather than the run's whole deadline. Also drops 6s of `sleep` from the suite.
ND-21, measured 2026-08-23. D-k's argument was corrected once already — from "a relay serves one network" to "on a private-domain grid, grid IS domain, structurally". The second half is false too, and it is false on the supported topology. grid-apis `store.member_for_access` checks an allowlist row BEFORE anything else, on every network type, and grid-src `grid_auth._requires_allowlist` names three ways onto a private-domain grid — right domain, allowlist row, or owning the grid — carrying the sentence "an invited account from another domain keeps working after the switch". An invited outsider is therefore authenticated, so D-k gives them every non-private project on the grid, imported company repositories included, plus a `project_members` row minted on their first read that needs a key. The relay does not compare domains and cannot: `grid_auth.GridAuthContext` carries `email`; `relay.AuthContext`, the object a route is handed, does not. The premise is enforced entirely at the control plane, and the rule there ADMITS rather than restricts. Decided: correct the claim, do not narrow the gate. The premise now reads "authenticated on this grid => a colleague OR somebody a colleague invited", and the blast radius is accepted with it — the alternative asks the control plane for a new value saying WHY a caller was admitted, which is a lockstep value and a rollout order for a boundary that is already live; narrowing would revoke access grid-apis explicitly promises keeps working; and `visibility` already exists as the instrument. What is NOT accepted is a justification standing while being false. In fairness to the shape being accepted: the minted row IS visible afterwards in `grid project member list`. Nobody is notified, and access is granted before anyone looks. Pinned from the other side by grid-src `test_project_visibility.TestAnInvitedOutsiderIsAColleagueToThisRule` — a characterization test, so a future narrowing announces itself in the ADR rather than in a support ticket. Narrowing is follow-up work, not a defect against this decision.
ND-16/F-3, the CLI half. The relay now answers `changed_since_count` and
`changed_since_paths` on `GET /tasks/{id}`; `grid task get` turns them into a
sentence, the file names, and the one command that shows what is there now.
The failure it closes is a task that reads like a success everywhere. The grid
applies every finished task to the project itself, and when two collide the step
that combines them runs in the OTHER person's conversation — where the decision
to drop somebody's work is recorded, and where they cannot read it, because
`grid task list --project` returns only the asker's own turns. Measured live
2026-08-20: `state=completed`, `grid task diff` showing their own change, and
the project holding somebody else's line.
Both keys, both type-checked, or it says nothing. Absence is what every relay
deployed before this sends, so a missing or unreadable pair must read as
"nothing to show" and never as a warning invented client side — the rollout is
relay before CLI, so that is the ordinary state of every grid for its duration.
`0` is the relay's positive answer that the work still stands, and it is the
common case: a note that appeared on every task would be furniture within a day.
`bool` is refused explicitly, since it is a subclass of `int` and `True` would
otherwise print "1 of the files ...".
The path in the suggested command goes through `shlex.quote`: a repository that
arrived through `import` holds whatever names its author gave it, and a warning
about lost work whose one next step does not run is worse than none.
A drift on the relay side is SILENT in the direction that costs somebody their
work — the CLI prints nothing for a renamed key exactly as it does for an old
relay — so `tests/test_task_lease.py` pins both names against grid-src's
`get_task`, and CLAUDE.md's register carries the row. It is pinned on `get_task`
and deliberately not on `task_view`: the answer costs two git reads, and
`task_view` also builds every row of `GET /tasks`.
`docs/cli.md` shows the output and says what `--json` carries.
B7.2. Two states look identical from outside — a queue that is not moving — and they want OPPOSITE actions. Nobody serving means find another machine; everybody serving and out of subscription headroom means wait, and a second subscription on the same account would not have helped. Sending a team to `grid join` for the second is the expensive kind of wrong. The distinction is two branches of `_print_providers`, and nothing held them apart. Measured by moving the `grid join` line into the withdrawn sentence: all three existing cases stayed GREEN, because each asserts what IS said and none asserts what must not be. The new pair closes it from both ends — the paused sentence must name the headroom and must NOT name `grid join`, and the fleet with nobody online must name it, or the first case would pass against a build that gives no advice at all.
…eased E4.3, and what it pins is not what the scenario is named for. The scenario asks for three held workspaces against a cap of one, expecting the answer *three remain* to be the discriminator. Measured: it is not. Under a sweep that wrongly counts a skip as progress the answer is still three, because there is nothing colder to evict — the fault is invisible in this shape, and the two existing cases catch it precisely because they have a cold workspace to lose. What no other case catches is a `release()` for a reservation the sweep never took. That is the expensive one: eviction takes the SAME reservation a worker takes (`tasks._reserve_workspace`) rather than reading a second registry, so a spurious release hands away a workspace somebody is running a turn inside. One registry, one owner, and this is what keeps it that way. Mutation: adding `release(triple)` to the skip branch kills this case and nothing else. The `asked` count is the harness's own control — *three remain* is also what a sweep that never looked at anything produces, which is how a bound that has quietly stopped working reads. E4.4 and E4.6 needed nothing: moving the sweep above `ensure_workspace` kills three cases, two of them pre-existing, and removing the per-store lock kills `test_conversations_of_one_member_materialize_at_the_same_time_into_one_store`. A case written for E4.4 here was duplication and was deleted rather than kept.
`GRID_TASKS` and `GRID_MAX_TASKS` were read inside `remote/serve.py`, the serve child's own module. The CLI parent needs the same two answers before it spawns that child (issues 58, 60, 61) and cannot ask `serve` for them — so without this it would read them for itself, and two readings of one rule get edited apart while every test stays green. `remote/task_opt_in.py` holds both, with their reasoning intact: why the opt-in is read at serve time rather than baked into the run record, and why the worker count has deliberately no upper clamp. `tests/test_task_opt_in.py` enforces the one-reading rule by parsing the source rather than by convention. It measures a READING, not a MENTION: `task_evict` names `GRID_MAX_TASKS` in its docstring to say what bounds what, so the scan inspects string literals in code with docstrings excluded, and matches the whole name — `GRID_TASK_ROOT` shares a prefix and a substring match reported the workspace root as a second opt-in reading. No behaviour change. 3135 passed, 9 skipped.
Everything that can be wrong in a provider's task configuration was checked only after a task had been claimed — which is after a member is already waiting. `preflight()` and `resolve_binary()` had one call site each, both inside `run_task`. So the join printed "serving", `grid project status` said online, and the provider looked healthy until somebody else's task died. Not a documentation gap: the environment table was already complete. The wrong moment. Issues 22 and 23 made these failures loud; they were loud to the wrong person. `task_agent.preflight_before_serving()` asks the same two functions from the CLI parent — never a copy of them — plus one question asked nowhere else: whether this account could create a workspace under GRID_TASK_ROOT at all. That one is unconditional, because an unwritable root fails every task with the sandbox on or off. It probes and never creates: which directory the root should be is issue 62's decision. The parent is the only place the sentence can be heard. The serve child is detached with both streams redirected into the engine log, and the parent tails that log only when the child died — so a refusal printed there, which must not kill the child, lands in a file nobody opens. A failing check does not fail the join. The provider serves inference, the answer names the fix and `grid join --respawn`, and the child is spawned with the opt-in withheld so it claims nothing it cannot finish. Measured on macOS: GRID_TASK_ROOT=/var/grid-fresh-provider is refused naming /var, and a writable root is allowed. 3148 passed, 9 skipped.
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
Seven commits on the task plane. The headline is issue 58: every check of a provider's task
configuration used to run only after a task had been claimed — which is after a member of the
grid is already waiting.
grid joinprinted "serving",grid project statussaid online, and theprovider looked healthy right up until somebody else's task died on it.
Not a documentation gap: the environment table in
docs/cli.mdwas already complete. The wrongmoment.
The commits
e300d1ce9c1200dd7dcb4changed_sincepair43535f0e1835f60d8c3948ec0494grid joinsays why it will not serve tasks (issue 58)Issue 57 — one home for the task opt-in
GRID_TASKSandGRID_MAX_TASKSwere read insideremote/serve.py, the serve child's own module.The CLI parent needs the same two answers before it spawns that child, and cannot import
serveforthem — so without this it would read them for itself, and two readings of one rule get edited apart
while every test stays green.
remote/task_opt_in.pyholds both, with their reasoning intact.tests/test_task_opt_in.pyenforces the one-reading rule by parsing the source, not by convention — and it measures a
reading, not a mention:
task_evictnamesGRID_MAX_TASKSin its docstring to say what boundswhat, so the scan inspects string literals in code with docstrings excluded, and matches the whole
name (
GRID_TASK_ROOTshares a prefix, and a substring match reported the workspace root as asecond opt-in reading).
No behaviour change.
Issue 58 —
grid joinsays why it will not serve taskstask_agent.preflight_before_serving()asks the same two functionsrun_taskasks — never acopy of them, so no second opinion about the sandbox, the permission mode or the version floor
exists anywhere — plus one question asked nowhere else: whether this account could create a
workspace under
GRID_TASK_ROOTat all. That one is unconditional, because an unwritable root failsevery task with the sandbox on or off. It probes and never creates.
A failing check does not fail the join. The provider serves inference, the answer names the fix
and
grid join --respawn, and the child is spawned with the opt-in withheld so it claims nothing itcannot finish. Refusing the whole join would take a working inference provider down over a task
misconfiguration — including for the operator who keeps
GRID_TASKS=1in a shell profile.detached with both streams redirected into the engine log, and the parent tails that log only
when the child died. Since a task-serving refusal must not kill the child, anything printed there
lands in a file nobody has a reason to open. The parent is the only place the sentence can be heard.
Test plan
.venv/bin/pytest tests/— 3148 passed, 9 skipped (after mergingmaindown).venv/bin/ruff check .— cleanGRID_TASK_ROOT=/var/grid-fresh-provideris refused naming/var; a writable root is allowedthe join has acted, narrowing
except (Exception, SystemExit)toexcept Exception, andadding a version floor to the new entry point are each killed by the test that claims to cover
them
mainmerged down into the branch first, so what lands is what was tested (README-only, noconflict)
tests/test_task_lease.pySKIP unless grid-src sits beside thisworktree — they skip in CI, and were run locally against the matching grid-src branch
Rollout
Nothing in these seven commits adds a cross-repo lockstep value.
dd7dcb4is the CLI half ofthe relay's
changed_sincepair — relay-first, and that relay is already deployed, so this is steptwo of that rollout.