Skip to content

Point the e2e CLI at the server under test, and roots at the dock - #290

Merged
pcarrier merged 2 commits into
mainfrom
claude/fix-e2e-main
Aug 17, 2026
Merged

pcarrier merged 2 commits into
mainfrom
claude/fix-e2e-main

Conversation

@pcarrier

@pcarrier pcarrier commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Why

CI's e2e job has been red on main since e149dfc (2026-08-16). Two specs fail deterministically, on every PR, so nothing can go green — including #289, whose only failing check is this one.

Both failures are stale tests, not product regressions.

panel-subscriptions — the oracle was a different server

The spec shells out to blit client list. start-servers.sh puts its server on a mktemp -d socket and exports BLIT_SOCK to its own process tree only, so the CLI resolved the default socket instead, started a server of its own there, and read that empty one's client list. Hence Expected length: 2 / Received: [] — the browser's subscriptions were on the other server all along.

It passes locally because reuseExistingServer hands back the developer's already-running gateway, and that one does front the default server.

The script now publishes its socket path in e2e/.e2e-socket for as long as its servers live; the spec reads it and passes BLIT_SOCK to the CLI. With no file (someone else's gateway) the CLI's own resolution is still correct, and a stale file whose socket is gone is ignored rather than used to start a third server.

roots-entry — asserted an entry that was deliberately removed

SwitcherOverlay.tsx says it outright: palette, font, workspace roots and remotes are not switcher entries, because the status bar and left dock already reach them in one click. The spec still asserted a Workspace roots row in Cmd+K. It now holds the app to the contract it actually has — the ⚙ beside the workspace-root selector opens the dialog, and nothing else does — and the comment in createKeyboardShortcuts.ts that still advertised the switcher entry is corrected.

Also

playwright.config.ts gains the CHROMIUM_BIN escape hatch playwright.dev.config.ts already had, so this exact suite can be reproduced outside CI on NixOS (the npm-bundled browser cannot find libglib there). Inert in CI.

Verification

Full suite, locally, CI=1 with an isolated XDG_CONFIG_HOME (so the run sees one remote, as CI does):

  • before: 10 failures, including both of these
  • after: both pass; 5 consecutive pair runs green

The other local failures (composition-chip, font-size-sync, text-prediction) fail identically with and without this change and do not reproduce in CI — they are this machine, not the branch.

@indent

indent Bot commented Aug 17, 2026

Copy link
Copy Markdown
PR Summary

Fixes the two e2e specs that have kept CI's e2e job deterministically red on main since 2026-08-16, blocking all merges. Both were stale tests rather than product regressions. (Note: the branch was force-pushed and rebased onto main, dropping the four unrelated media commits that were present in an earlier head; the PR is now purely e2e/test changes.)

  • panel-subscriptions.spec.ts: start-servers.sh runs its server on a private mktemp socket exported only to its own process tree, so the CLI resolved the default socket and read an empty server's client list. The script now publishes its socket path to a new gitignored e2e/.e2e-socket, and the spec reads it to point blit at the right server, with fallbacks for a reused developer gateway (no file) and a stale/dead socket. Now that the CLI reaches the real server the spec's two cat sessions are live, so an afterAll(closeAllTerminals) was added to tear them down — a leftover focused cat was echoing typed input and failing later text-prediction specs.
  • roots-entry.spec.ts: rewritten to assert the actual contract — workspace roots opens from the ⚙ beside the dock's workspace-root selector and is deliberately not a Cmd+K switcher entry.
  • playwright.config.ts: adds the CI-inert CHROMIUM_BIN escape hatch (already in playwright.dev.config.ts) so the suite runs on NixOS.
  • createKeyboardShortcuts.ts: corrects a stale comment that still advertised the removed switcher entry.

Issues

Review closed.

View session

The e2e job has been red since roots moved out of the Cmd+K switcher: two
specs assert contracts the app no longer has.

`panel-subscriptions` reads `blit client list` as its oracle, but
`start-servers.sh` puts its server on a private socket and exports it to
nothing but the gateway. The specs run in another process tree, so the CLI
resolved the *default* socket, started a server of its own there, and
reported that empty one's client list — zero subscriptions, forever. It
passes locally only because `reuseExistingServer` hands the developer's own
gateway back, and that one does front the default server. The script now
publishes its socket path in `e2e/.e2e-socket` for as long as its servers
live, and the spec passes it to the CLI as BLIT_SOCK; with no file, the CLI's
own resolution is still the right answer.

`roots-entry` asserted a "Workspace roots" entry in the switcher, which is
deliberately gone: the switcher carries only what the chrome cannot reach,
and the ⚙ beside the workspace-root selector opens the dialog in one click.
The spec now holds the app to that, including the absence of a second global
affordance, and the stale comment in createKeyboardShortcuts.ts that still
advertised the entry is corrected.

`playwright.config.ts` gains the CHROMIUM_BIN escape hatch its dev sibling
already had, so this exact suite can be run outside CI on NixOS, where the
npm-bundled browser cannot start.

Co-Authored-By: Claude <noreply@anthropic.com>
@pcarrier
pcarrier force-pushed the claude/fix-e2e-main branch from f462f90 to f43740f Compare August 17, 2026 22:49
@github-actions

Copy link
Copy Markdown

🔗 Preview: https://blit-ly6lg2p4h-indent.vercel.app

@github-actions

github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown

Coverage

Crate Lines Functions Regions
alacritty-driver 73.7% (840/1139) 77.5% (69/89) 77.1% (1363/1767)
browser 0.0% (0/825) 0.0% (0/69) 0.0% (0/1404)
cli 37.8% (5196/13744) 43.8% (511/1168) 38.6% (8077/20900)
compositor 34.9% (5961/17103) 49.3% (495/1005) 34.6% (8182/23649)
desktop 77.6% (4164/5366) 70.4% (367/521) 74.0% (5772/7804)
fonts 85.3% (756/886) 89.9% (71/79) 86.4% (1485/1719)
fssync 92.6% (5518/5961) 94.4% (501/531) 92.8% (10186/10981)
gateway 34.0% (669/1966) 40.2% (68/169) 31.4% (1029/3279)
git 87.6% (4478/5111) 89.6% (362/404) 87.3% (7130/8163)
guest 83.2% (2211/2658) 80.6% (250/310) 81.9% (3708/4529)
lsp 77.2% (2688/3483) 79.4% (262/330) 74.9% (4210/5619)
proxy 19.2% (172/898) 20.5% (26/127) 21.0% (293/1392)
remote 91.4% (15583/17054) 94.2% (1082/1149) 88.7% (25259/28474)
sd-notify 72.8% (67/92) 100.0% (6/6) 82.4% (108/131)
server 55.9% (30939/55362) 63.1% (2656/4209) 57.2% (46915/81960)
ssh 32.2% (165/512) 48.2% (27/56) 31.4% (261/830)
upsidedown 31.4% (391/1247) 27.8% (55/198) 34.8% (797/2287)
webrtc-forwarder 8.5% (238/2805) 10.7% (22/205) 6.3% (289/4595)
webserver 64.4% (1250/1941) 67.6% (173/256) 66.6% (2099/3151)
Total 58.8% (81286/138153) 64.4% (7003/10881) 59.8% (127163/212634)

@blacksmith-sh

blacksmith-sh Bot commented Aug 17, 2026

Copy link
Copy Markdown

Found 12 test failures on Blacksmith runners:

Failures

Test View Logs
[chromium] › tests/
mobile-keyboard.spec.ts:200:5 › a tap while the keyboard failed to rise retries instead
of hiding
View Logs
[chromium] › tests/
pane-tools.spec.ts:311:7 › Parked terminal does not resurrect › closing the session tha
t displaced a parked one shows it, not an empty pane
View Logs
[chromium] › tests/
text-prediction.spec.ts:102:7 › host text prediction › a Backspace through the field de
letes one character, not none or two
View Logs
[chromium] › tests/
text-prediction.spec.ts:136:7 › host text prediction › a proposal shows in a chip on th
e cursor's line and is not forwarded
View Logs
[chromium] › tests/
text-prediction.spec.ts:184:7 › host text prediction › with prediction off the field st
ays empty, as it always was
View Logs
[chromium] › tests/
text-prediction.spec.ts:68:7 › host text prediction › the session runs a shell that edi
ts its own line
View Logs
[chromium] › tests/
text-prediction.spec.ts:88:7 › host text prediction › typed characters reach the pty ex
actly once
View Logs
host text prediction/
a Backspace through the field deletes one character, not none or two
View Logs
host text prediction/
a proposal shows in a chip on the cursor's line and is not forwarded
View Logs
host text prediction/the session runs a shell that edits its own line View Logs

...and 2 more test failures. View all on Blacksmith

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

Now that its CLI reaches the server under test, the spec's two `cat`
sessions are real, and it left them running. The next spec to type a command
into the focused terminal typed it into `cat`, got its own text back, and
reported the feature under test as broken — five text-prediction failures
whose oracle is a file the shell never wrote.

Co-Authored-By: Claude <noreply@anthropic.com>
@pcarrier
pcarrier merged commit 01c86af into main Aug 17, 2026
10 checks passed
@pcarrier
pcarrier deleted the claude/fix-e2e-main branch August 17, 2026 23:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant