You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Test (coding-agent 1/3) fails at main: refinement.test.ts row "ranks by 'recency as the tie-break for equal scores'" expects [global:zz_newer] to win an equal-score tie, but that expectation is stale after #2400.
Two independently-green PRs combined into one red expectation:
[RSI, other] Rank digest and harness search by IDF-weighted terms #2392 (wave 2, its main-tip CI was cancelled, never verified) rewrote this row's fixture to aa_older/zz_newer with IDENTICAL titles, so the row exercises the then-current recency tie-break in compareRankedHarnessEntries. Alone (7f39eb8): all 76 tests pass.
[RSI, performance] Stabilize continual-harness prompt material for prompt-cache hits #2400 (wave 3) deliberately replaced the recency tie-break with stable identifier order ([path, title, id]) to keep the rendered digest prefix stable for provider prompt-cache hits, and added the opposite-pinning test "breaks score ties by stable identifier order, not recency". Alone on its branch: all tests pass — but only by an ICU collation accident: the pre-[RSI, other] Rank digest and harness search by IDF-weighted terms #2392 version of this row used titles "Worktree policy" vs "Worktree policy 2", and localeCompare ignores the \0 join separator while the digit "2" sorts before letters, so "newer" won accidentally.
Combined at main: identical titles + identifier-order tie-break -> aa_older renders -> the stale zz_newer expectation fails.
No harness entry is lost from the built digest: this is the tie-break policy between two equal-score entries, and the merged #2400 contract (stable order, by design) is what the digest follows.
Fix
Delete the stale row (11 lines). It is now a true duplicate of #2400's merged test "breaks score ties by stable identifier order, not recency", which pins the same observable (equal scores, same titles, window 1 -> identifier order beats recency) and passes under #2392's IDF scoring. Updating the row instead would only create a second copy of that vector.
Validation
refinement.test.ts: 76/76 green (was 1 failed / 76 passed).
suite/agent-session-prompt.test.ts + suite/agent-session-compaction.test.ts: 69/69 green; recursion digest test green.
Test-line budget: net -11 test lines (deletion only), gate passes.
Authorization: fleet-manager fix-forward, authorized by the session owner's delegate (faerie1) under the standing 'until all are merged' directive: the main-red was caused by the wave-3 merge interaction of #2392 and #2400 (wave-2's main-tip CI was cancelled, never verified against #2400's file state). No functional regression: the deleted row is a true duplicate of #2400's merged tie-break test. Trivially revertable if overruled.
Note
Low Risk
Test-only deletion; harness ranking implementation is unchanged.
Overview
Removes a harness digest ranking table row in refinement.test.ts that still expected recency to break equal relevance scores (zz_newer over aa_older).
That expectation is obsolete after the merged tie-break policy uses stable identifier order (path, title, id) for prompt-cache stability, not updated_at. The same behavior is already asserted by "breaks score ties by stable identifier order, not recency", so deleting the row fixes the failing main CI without changing production ranking logic.
Reviewed by Cursor Bugbot for commit 78fcc36. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Drop stale recency tie-break test in refinement.test.ts harness digest ranking
Removes the test case that asserted the newer worktree entry wins a recency tie-break over alphabetical ordering when relevance scores are equal. The expectation contradicts behavior established in #2400.
Main resolved at 2026-09-18T00:35:48.601923+00:00. Harness 126fe701.
Linux x64, 4 vCPU, 8 GB RAM, 20 GB disk; region us.
Image: node:24-bookworm@sha256:be23f54a88d34e8824c741b19b91064094f92c1c97b194144bfc8b50d67258e2.
Stock tools, skills, daemon, and Python bootstrap enabled; fresh homes and a fixed Git fixture.
Onboarding is dismissed; the editor starts without a selected model or submitted prompt.
Medians shown. Arrows require a 20% timing/memory change plus absolute floors and IQR.
These practical noise floors are not a statistical significance test.
Cold means stopped Prime processes; OS filesystem caches are not flushed.
No model requests or credentials. Installation excludes build/setup time.
Installer tarballs use loopback; npm/Python downloads use the network with fresh caches.
Artifact size counts release tarballs; footprint after first use includes registry packages.
MB is decimal. Summed RSS can double-count shared pages; PSS is recorded when available.
Provisioning, setup, and build durations are recorded separately in the raw results.
Kernel probes use the installed JSONL runtime, outside the TUI/TypeScript host.
Per trial: 50 Python cells, 5 calls per shell case, and one 35-cell mix (9 git status calls).
Cell/shell values are batch means; other runtime timings are single operations.
State fixture: a 10,000-row × 8-column integer DataFrame and a 10,000-integer list.
Restore runs in a fresh kernel, including pandas imports; kernel startup is excluded.
Kernel RSS covers the isolated Python process; loaded RSS follows the pandas workload.
Transport benches run node against the prepared source build, outside the installed home.
Frame decode times one 32 MiB private frame, snapshot-chunk header, pushed in
8 KiB chunks; the wire shape of multi-MB frames on the daemon-worker channels.
UI trials use a fresh fixture set: 194 top-level sessions including one ~40 MB transcript,
40 ledger fan-out children, and a 6-deep subagent chain (~46 spawn edges).
Large fixtures hold 1,999 complete triples (~5 MB JSONL); medium 119; subagents 399 each.
Interactions: cold --resume of a large session, warm /resume switch, left-arrow to agents view,
roster settle with many saved sessions, search-and-open of another large session,
reattaching to that resident session, opening the chain parent, and drilling to depth 6.
Readiness is the rendered transcript tail plus a confirmed editor echo.
CPU metrics sum utime+stime across the whole benchmark-user process tree per interaction.
UI memory sums RSS after the interactions; PTY byte counts are in the raw results.
A separate catalog fixture has 2,300 sessions, 2,298 edges, and 13 paused scheduled-job owners.
Catalog timings cover first/repeated reads and cold worker creation under three pending scans.
All expected jobs and owner metadata are checked; worker readiness excludes TUI rendering.
Costs estimate full sandbox lifetimes at configured rates, including setup and build.
Budget target: $1; not a billing cap. Performance changes are informational.
Failed or incomplete execution fails the workflow; saved artifacts remain available.
Each side stops a phase after 2 identical consecutive failures.
Skipped trials are not attempted samples. Warm startup requires a successful cold launch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why main is red
Test (coding-agent 1/3)fails at main:refinement.test.tsrow "ranks by 'recency as the tie-break for equal scores'" expects[global:zz_newer]to win an equal-score tie, but that expectation is stale after #2400.Two independently-green PRs combined into one red expectation:
aa_older/zz_newerwith IDENTICAL titles, so the row exercises the then-current recency tie-break incompareRankedHarnessEntries. Alone (7f39eb8): all 76 tests pass.[path, title, id]) to keep the rendered digest prefix stable for provider prompt-cache hits, and added the opposite-pinning test "breaks score ties by stable identifier order, not recency". Alone on its branch: all tests pass — but only by an ICU collation accident: the pre-[RSI, other] Rank digest and harness search by IDF-weighted terms #2392 version of this row used titles "Worktree policy" vs "Worktree policy 2", andlocaleCompareignores the\0join separator while the digit "2" sorts before letters, so "newer" won accidentally.aa_olderrenders -> the stalezz_newerexpectation fails.No harness entry is lost from the built digest: this is the tie-break policy between two equal-score entries, and the merged #2400 contract (stable order, by design) is what the digest follows.
Fix
Delete the stale row (11 lines). It is now a true duplicate of #2400's merged test "breaks score ties by stable identifier order, not recency", which pins the same observable (equal scores, same titles, window 1 -> identifier order beats recency) and passes under #2392's IDF scoring. Updating the row instead would only create a second copy of that vector.
Validation
refinement.test.ts: 76/76 green (was 1 failed / 76 passed).suite/agent-session-prompt.test.ts+suite/agent-session-compaction.test.ts: 69/69 green; recursion digest test green.Authorization: fleet-manager fix-forward, authorized by the session owner's delegate (faerie1) under the standing 'until all are merged' directive: the main-red was caused by the wave-3 merge interaction of #2392 and #2400 (wave-2's main-tip CI was cancelled, never verified against #2400's file state). No functional regression: the deleted row is a true duplicate of #2400's merged tie-break test. Trivially revertable if overruled.
Note
Low Risk
Test-only deletion; harness ranking implementation is unchanged.
Overview
Removes a harness digest ranking table row in
refinement.test.tsthat still expected recency to break equal relevance scores (zz_neweroveraa_older).That expectation is obsolete after the merged tie-break policy uses stable identifier order (
path,title,id) for prompt-cache stability, notupdated_at. The same behavior is already asserted by "breaks score ties by stable identifier order, not recency", so deleting the row fixes the failing main CI without changing production ranking logic.Reviewed by Cursor Bugbot for commit 78fcc36. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Drop stale recency tie-break test in
refinement.test.tsharness digest rankingRemoves the test case that asserted the newer worktree entry wins a recency tie-break over alphabetical ordering when relevance scores are equal. The expectation contradicts behavior established in #2400.
Macroscope summarized 78fcc36.