Skip to content

fix(cli): persist what the server answered for a captured page - #3553

Open
miguel-heygen wants to merge 2 commits into
mainfrom
feat/capture-persist-http-status
Open

fix(cli): persist what the server answered for a captured page#3553
miguel-heygen wants to merge 2 commits into
mainfrom
feat/capture-persist-http-status

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Aug 30, 2026

Copy link
Copy Markdown
Collaborator

The failure

hyperframes capture against a URL that answers 404 completes successfully. The error page has a title, a palette, typefaces, sections and a DOM, so every extractor downstream reads it happily, and the capture returns ok: true with a full design token set. Nothing in the output says the server refused.

Reproduced against a local server that answers 404 with a styled error page — 11 colours and 4 typefaces extracted, lastPhase: complete:

$ hyperframes capture http://127.0.0.1:8931/anything -o out --json
  "ok": true,
  "title": "404 Not Found",
  "lastPhase": { "phase": "complete", "status": "completed" }

$ python3 -c "..."   # read out/extracted/tokens.json
colors : 11 ['#FFFFFF', '#000000', '#24292F', '#F6F8FA', '#CF222E', ...]
fonts  : 4 ['Georgia', 'Courier New', 'Verdana', 'Mona Sans']

The status was already read, once, to feed detectBlockedPage, and then dropped on the floor. A consumer of a capture directory therefore had no way to learn it.

Why not widen detectBlockedPage

Adding 404 to its status set looks like a one-line fix and is the wrong one. That helper answers "does the rendered document look like an access-protection wall?" — a heuristic gated on a minimal DOM, which a rich error page fails anyway. "Was this blocked?" and "what did the server answer?" are two different questions, and making the heuristic the carrier for the second leaves a fact owned by a guess.

The change

The status is persisted plainly, as its own record:

  • extracted/response.json{ "status": 404 }, written from the same value detectBlockedPage already receives.
  • CaptureResult.httpStatus, so an in-process caller does not have to read a file the function just wrote.
  • httpStatus in capture --json, the documented programmatic surface. Leaving it out would repeat the same read-once-and-discard one boundary later: an agent reading ok: true off a capture of a 404 could not see it.

Written before the blocked-page check, so the record's absence means "navigation never produced a response" — a third state, distinct from a status of 404 and from a status of null. Those three are not collapsed: null is "we never learned what the server said", which is not "fine".

No behaviour changes here. Deciding what a non-success response means is left to each consumer, which is why this ships as a fact rather than a refusal.

Verification

  • bun run --cwd packages/cli test — 197 files, 2867 passed, 3 skipped, 0 failed.
  • bun run --cwd packages/cli typecheck — clean (after packages/core build).
  • oxlint / oxfmt --check on the changed files — clean.
  • Real capture, real Chrome: the 404 above persists {"status": 404}; the same page served as 200 persists {"status": 200}.
  • The new test is non-vacuous: coercing null to 0 in the writer fails it with expected +0 to be null; restored byte-exact and it passes again.

A page that renders is not a page that succeeded. An error page has a title,
a palette, typefaces and a DOM, so every extractor downstream reads it happily
and produces a design system belonging to whoever wrote the error page rather
than to the site's owner.

The status was already read, once, to feed `detectBlockedPage`, and then
dropped. That helper cannot stand in for it: it decides whether the rendered
document LOOKS like a protection wall, over a minimal-DOM heuristic that a
rich error page passes. "Was this blocked?" and "what did the server answer?"
are two questions, and widening the first to carry the second would leave a
heuristic owning a fact.

So the response status is persisted plainly, as its own record, and every
consumer decides for itself what a non-success response means for its product.
Written before the blocked-page check runs, so the record's absence means
"navigation never produced a response" — a third state distinct from a status
of 404 and from a status of null.
…tput

The status reached `CaptureResult` and was then dropped at the CLI boundary,
which is the same read-once-and-discard that made the error page harvestable
in the first place. `--json` is the documented programmatic surface, and an
agent reading `ok: true` off a capture of a 404 has no way to see it there.

@jerrai-bot-heygen jerrai-bot-heygen left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at c3f279d299e299170882fc0a40bda07379caf216.

response.json, the in-process result, and the JSON CLI output preserve the same final navigation status without conflating null, 0, and a response status. The record is written before the independent blocked-page heuristic, so callers retain the server fact even when the capture produces a rich non-success document. Focused CLI/type/lint checks are green and no inline threads are open.

— Jerrai

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants