fix(#5970): size the daemon's in-process parse gate from the background core budget

[cajasmota]

The daemon's in-process tree-sitter parse gate was sized from runtime.GOMAXPROCS(0) — on a 12-core host, a cap of 12. Background indexing is supposed to get 25% of the machine (process.IndexCoreBudget(), max(1, NumCPU/4)), so the gate that is meant to bound it was running at 4x the policy. This is write-side RSS: each concurrent parse holds a tree-sitter tree, and tree-sitter allocates through cgo, which GOMEMLIMIT cannot see.

The gate is now sized env > cpu.json > 25% budget — the same precedence chain resolveDaemonGOMAXPROCSWith documents, differing only in the trailing default. On a 12-core host: 12 → 3.

Precedence, and a correction

cpu.json is now consulted. An earlier draft of this description called that a restored behaviour, on the theory that the old runtime.GOMAXPROCS(0) read reflected the pin because applyDaemonGOMAXPROCSFromCaps had already applied it. That was wrong, and worth recording rather than quietly fixing: applyDaemonGOMAXPROCSFromCaps has exactly one non-test call site, inside the SIGHUP handler (cmd/grafel/daemon.go:394). Nothing calls it at startup. So at process start the runtime carried the env-or-half-cores value and the cpu.json pin was already ignored by the parse gate.

An operator pinning the daemon to 1 core via cpu.json on a 64-core box therefore got a parse gate of 12 before this change, not 1. This closes a pre-existing gap; it does not restore something that previously worked. installCapReloadHandler now re-resolves the gate on SIGHUP alongside the GOMAXPROCS re-apply, so the live-mutable surface stays live for the gate that actually bounds cgo.

Dropping the half-cores GOMAXPROCS default is deliberate: internal/process/corebudget.go:38-45 already documents that GOMAXPROCS cannot bound cgo, so deriving a cgo-allocation cap from it is a category error.

The foreground lift was suspending the budget for the whole rebuild

indexstate.BeginForegroundParse lifts the cap process-wide, not for the calling unit of work: effectiveParseCapLocked returns 0 for every acquirer while any hold is live, and there is no caller identity in the gate. A probe measured 20 concurrent background parses against a background cap of 2, and because release does not preempt, the overshoot persists past the release until those parses drain.

daemonRebuildFuncCore held it across the entire rebuild — self-documented elsewhere as 10-12 minutes — while the barge taken alongside it gates EXCLUSIVE stages only and explicitly does not stop watcher-driven incremental indexing.

And on the shipped default that hold bought nothing. With the subprocess indexer on, the rebuild does no in-process tree-sitter parsing at all: the per-repo index forks via runRebuildSubprocess, the in-process indexFn is the GRAFEL_SUBPROCESS_INDEXER=0 branch, and internal/links has no treesitter dependency. So the hold was pure downside — a multi-minute window in which the budget this commit exists to enforce was not enforced, in exchange for nothing.

The hold is now conditional on !sched.SubprocessIndexEnabled(). The BeginForegroundParse doc comment previously read as though the exemption were scoped to the caller ("a USER-AWAITED unit of in-process parsing"), which is how this call site came to exist; it now opens with "READ THIS BEFORE ADDING A CALL SITE", states the process-wide semantics and the non-preempting release, and carries the probe numbers.

The per-caller exemption (AcquireParseSlotForeground, bypassing the semaphore without mutating shared parseGateCap) is the correct end state and is not built here — see #6022. The residual escape is currently bounded to ≤2 by the scheduler worker pool being hard-coded to 2 and TryIncremental extracting sequentially; note that raising that pool for throughput would silently turn this back into a memory regression.

Verification

The cap binds on the path that runs: installDaemonParseCap sits in the shared prelude of runDaemonMode, ahead of the mode branch, so both grafel serve and grafel engine route through it. Nothing re-installs downstream — ensureParseConcurrencyDefault early-returns because the configured cap is always > 0 in the daemon, and extract-internal is a separate process. The ParseConcurrencyCap / EffectiveParseConcurrencyCap split is load-bearing: had ensureParseConcurrencyDefault read the effective cap, a grafel index RPC inside a daemon running GRAFEL_DAEMON_GOMAXPROCS=12 would clobber 12 → 3.

runDaemonMode is not directly testable, so the install is pinned by an AST assertion. Review found the first version of that pin value-blind — it checked only that the call existed, and installDaemonParseCap(runtime.NumCPU() * 4, ...) left all 418 tests green, reinstalling the original bug's exact magnitude. The pin now checks the argument expressions. Verified independently: that mutation now fails with runDaemonMode passes hostCPU=runtime.NumCPU() * 4 ... want runtime.NumCPU().

Full mutation battery, all died: call site reverted to runtime.GOMAXPROCS(0); helper GOMAXPROCS-shaped; scaled hostCPU argument; nil caps store; policy ignoring cpu.json; either foreground hold dropped; the lift made inert (this last one turns 4 tests red across two packages — a permanently-uncapped daemon is caught).

Values verified against indexCoreBudgetDivisor = 4: 1→1, 2→1, 4→1, 8→2, 12→3, 16→4, 32→8, 64→16, with the env override unclamped in both directions.

go build ./... && go vet && gofmt -l clean. 1369 tests across cmd/grafel, internal/daemon, internal/process, internal/indexstate. internal/daemon/sched -race -count=2 (394) and internal/indexstate + internal/treesitter -race -count=4 (540) green.

Known and bounded

Two edge cases accepted rather than fixed: --async on a watcher-less non-split daemon (s.scheduler == nil) falls through to the synchronous index path and takes the foreground lift — low severity, since there is no watcher in that configuration to escape; and in split mode the rebuild is applied asynchronously by the engine's rebuildWorker with retries, so the hold can be live with no human waiting, which is the same assumption the existing barge and foreground registry already make.

Independently adversarially reviewed.

Closes #5970
Refs #5954, #6022