You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Count a Bot repeating itself, and hand the count to the boundary (#17)
* Count a Bot repeating itself, and hand the count to the boundary
A model that cannot get something to work retries. It clicks the same button,
reloads the same page, writes the same file, and nothing counted it. Every one of
those attempts is a real action on somebody's live website and a real charge
against somebody's model credit, and the audit trail recorded them one row at a
time with no way to see that they were the same row thirty times over.
The gateway is the one place this can be counted, because it is the one place
every governed action already goes through and already writes a row for.
`computer/repeat.ts` keys a call on the tool plus the argument that says which
thing it acted on, over a sliding three-minute window, and the gateway asks it
before it asks the policy. The count lands in `PolicyContext` as `repeat.count`,
so a deployment can write `repeat.count >= 10` in its deny list and stop a Bot
going in circles with the boundary it already has, on the page it already uses.
The detector does not refuse anything, on purpose. Blocking here would be a
second boundary with rules of its own, invisible on the Boundaries page,
unanswerable to dry-run, and impossible to relax for the one Bot whose job really
is to poll something. It observes; the policy decides.
Crossing 3, 10, or 25 writes one `computer.action_repeated` row each, carrying
the tool, a readable fingerprint and the count. It is deliberately not filed as a
refusal: nothing was forbidden and nothing was stopped, and a trail that files an
observation as a refusal teaches a reader to skim past the refusals that are
real. The audit page gives it its own filter and its own words for the same
reason.
The window costs something and the preset says so. It is time-based, so a Bot
slow enough to spread its attempts wider than the window never trips it, and one
that varies a single argument each time round is ten different calls.
COMPUTER_REPEAT_WINDOW_MS widens it for a deployment whose provider is slow
enough to need that, and refuses to start rather than falling back, because a
rule about repetition that never fires looks exactly like a Bot behaving itself.
* Bound what the repeat detector remembers, and stop it dropping the key it is about to need
Three things the detector claimed and did not do.
The outer map was described as bounded by how many Bots a deployment has. It is
not: the id it counts against is the `:botId` in the request path, checked
against a session and against nothing else, because no acting route resolves it
to a row in `bots` first. A signed-in caller looping over invented ids bought a
map, an inner map and an occurrence record each time round, before the policy
was consulted and whether or not any such Bot existed, and none of it was ever
given back. Both levels are capped now and neither drops anything that is still
inside the window, so what is held is the recent past and nothing else.
Eviction was least recently seen, described as the key furthest from tripping
anything. For the one behaviour this feature exists to catch it is the nearest:
a Bot going round a loop of more distinct calls than the cap holds lost each key
exactly one step before it came back, so twenty rounds of a sixty-five step
circle reported every call as a first attempt and no rule about repetition could
fire. Nothing live is evicted at all now. A place is freed when a call ages out
of the window, and while every place is held by something still inside it, a
call the Bot has not made before is not counted rather than something live being
thrown out for it. The cost is a Bot whose first sixty-four distinct calls are
honest work and which only then gets stuck: its loop is invisible until one of
those falls out of the window, at most one window away. A blind spot that clears
itself is worth more than an eviction rule that can be wrong for as long as the
loop lasts, and it buys the other half: a call already being counted can no
longer be pushed out by a Bot doing other things in between.
The observational audit row blocked. `recordAuditEvent` rethrows, and that row
is written ahead of the policy, so a moment's trouble at the audit store refused
every third, tenth and twenty-fifth identical call, on an action nothing
objected to, from the one part of this that is not allowed to refuse anything.
It is swallowed and logged now. The invariant is untouched: the decision row
goes to the same store a few lines below, and a store that is really down stops
the action there.
Also honest about the costs where an operator sees them. The Boundaries preset
named only the two ways the rule under-fires and never the way it over-fires,
which is the failure that costs somebody their Bot: a Bot typing ten different
searches into one box is ten repeats, and the rule refuses the tenth. That
admission, the fact that the count lives in one process and so splits across
replicas, and the fact that MCP tool calls always report one, now reach the
preset, the `PolicyContext` docblock, `.env.example` and the architecture note.
The threshold docblock said a count that falls out of the window and climbs
again reports again; it is the window emptying completely that ends a run, which
is what the code has always done and what the type now says.
cost: "A password box the page labels something else is not covered, the rule matches the label.",
34
34
},
35
+
{
36
+
label: "Stop a Bot repeating itself",
37
+
// The count includes the attempt being decided, so this refuses the tenth, not the eleventh.
38
+
rule: "repeat.count >= 10",
39
+
cost: "Two calls count as the same call when the thing acted on is the same, whatever was typed into it, so a Bot running ten searches from one box, or reading one file ten times, is refused on the tenth. It misses the other way too: a Bot slow enough to spread its attempts wider than a few minutes is never caught, one that changes a single argument each time is ten different calls, and calls to another server's tools are not counted at all. Worth adding while a match is recorded and allowed, before it starts refusing anybody's work.",
0 commit comments