Skip to content

feat(api): validate every response against its Zod schema, and refuse a drift - #10435

Merged
JSONbored merged 1 commit into
mainfrom
feat/api-response-validation-only
Aug 10, 2026
Merged

feat(api): validate every response against its Zod schema, and refuse a drift#10435
JSONbored merged 1 commit into
mainfrom
feat/api-response-validation-only

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

Every REST response is now parsed against the Zod schema that defines it, on every request, and a response that does not match is not served.

Three things were wrong before:

  • It covered 5 of 161 routes. src/response-validation-tripwire.ts shipped in types-epic B: generate OpenAPI 3.1 from the Zod layer; invert the schemas/ flow for covered routes #7860 with a hand-written SCHEMA_LOADERS listing subnets, subnet-detail, health, economics and subnet-stake-quote, and a comment saying "add an entry here as later types-epic B batches convert more routes." The batches landed. The entries did not.
  • It was off. wrangler.jsonc set "METAGRAPH_VALIDATE_RESPONSES": "false", so the five did not run either.
  • It logged. console.warn on mismatch, then served the body anyway.

Derived, so there is no list to fall behind

A route names its artifact → schemaRefForArtifactPath maps that to the component id the OpenAPI document publishes → COMPONENT_SCHEMAS_BY_ID hands back the Zod node. That map is populated by register() itself — 322 components — so a route converted tomorrow is covered the moment its component is registered, which is the same moment it appears in openapi.json.

schemaRefForArtifactPath is exported from contracts.ts; the map is new and one line inside register.

It throws, and it is awaited

waitUntil was the wrong place the moment it started throwing: the response is already built by then, so a throw is an unhandled rejection and the drifted body ships regardless. It is awaited in the response path now, and both call sites turn ResponseSchemaDriftError into a 500:

{"error":{"code":"response_schema_drift", …}}

The tripwire's own failures — a broken import, an artifact with no contract entry — are still warned and swallowed. A drift is the route's fault and must fail the request; the tripwire falling over is not, and must not.

Cost, stated plainly: one parse per response while the flag is on, and a schema bug now fails a route instead of quietly shipping. That is the trade, and it is the reason the flag exists.

The one real disagreement it found

Sweeping the served surface for fields where a component claims non-null turned up exactly one where the producer disagrees: endpoint_pools.source is null in production against a required enum. Fixed at the Zod, with the evidence recorded inline — endpoint_pools(limit: 3) { source } answers "source": null today.

That matters beyond this route. The published GraphQL schema takes its nullability from these components and graphql-js enforces non-null at execution, so a single null where a component says non-null nulls the whole surrounding object and attaches an error — which is what SelfHealthLane.detail did on every self_health request (#10215). Until this ran, "the Zod says non-null" was a claim about the schema with nothing checking it against the producer. This is what keeps that count at one.

Tests

Twelve, covering both halves and both call sites:

  • a real, schema-matching envelope passes; a drifted one throws, carrying the route id and the parse error
  • the tripwire's own failure (mocked import blow-up) is warned, never thrown
  • an artifact with no contract entry is skipped, not failed — validate:openapi owns that invariant
  • an artifact whose component nothing registers warns once and is skipped, and the second call does not re-warn
  • coverage is derived: >300 components resolvable by id, and gaps — never one of the five pilots — validates cleanly against real fixture data
  • flag on, a drifting tripwire ⇒ 500 with response_schema_drift, asserted separately for handleApiRequest's generic path and for subnet-stake-quote, which is matched and returned early and so has its own call site
  • flag on, five real routes still answer 200

registerModuleStateReset is wired for the two module-level caches, so the warn-once does not leak across test files under isolate: false.

Validation

  • npm run typecheck, npm run lint, npm run format:check — clean
  • validate:contract-drift, validate:module-state-resets — pass
  • npx vitest run800 files, 18,483 passed, 11 skipped, 0 failed, with the flag ON for the whole suite
  • rebuilt on 8d3f84657

Refs #10214.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 10, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
metagraphed-registry-sync-api 859f040 Aug 10 2026, 12:29 PM

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 10, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
metagraphed-wss-lb 859f040 Aug 10 2026, 12:29 PM

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 10, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
metagraphed-data-api 859f040 Aug 10 2026, 12:29 PM

@superagent-security

Copy link
Copy Markdown

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Aug 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.78%. Comparing base (f8cd656) to head (859f040).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #10435   +/-   ##
=======================================
  Coverage   95.77%   95.78%           
=======================================
  Files         670      670           
  Lines       41987    42019   +32     
  Branches    15533    15540    +7     
=======================================
+ Hits        40212    40246   +34     
+ Misses        599      598    -1     
+ Partials     1176     1175    -1     
Files with missing lines Coverage Δ
schemas-src/openapi-registry.ts 100.00% <100.00%> (ø)
schemas-src/routes/endpoints-pools.ts 100.00% <ø> (ø)
src/contracts.ts 96.32% <ø> (+0.45%) ⬆️
src/response-validation-tripwire.ts 100.00% <100.00%> (ø)
workers/api.ts 91.33% <100.00%> (+0.02%) ⬆️
workers/request-handlers/entities.ts 89.55% <100.00%> (+0.05%) ⬆️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JSONbored JSONbored self-assigned this Aug 10, 2026
@JSONbored
JSONbored force-pushed the feat/api-response-validation-only branch 2 times, most recently from 246fafa to 47e1b38 Compare August 10, 2026 12:17
… a drift

The response tripwire shipped in #7860 as a five-route pilot: SCHEMA_LOADERS
hand-listed subnets, subnet-detail, health, economics and subnet-stake-quote,
with a comment saying to add an entry as later batches converted more routes.
The batches landed and the entries did not, so 156 of 161 routes served
unchecked -- and METAGRAPH_VALIDATE_RESPONSES was "false" in wrangler.jsonc, so
the five did too.

It is DERIVED now and covers everything by construction: a route names its
artifact, schemaRefForArtifactPath maps that to the component id, and
COMPONENT_SCHEMAS_BY_ID -- populated by register() itself, 322 entries -- hands
back the Zod node. There is no list to fall behind.

It THROWS, and is AWAITED in the response path rather than scheduled on
waitUntil, where a throw would only be an unhandled rejection and the drifted
body would ship anyway. A drift is a 500 the caller can act on.

This is also what the GraphQL cutover was missing. The published schema takes
its nullability from these components and graphql-js enforces non-null at
execution, so "the Zod says non-null" was a claim nothing checked against the
producer. A sweep of the served surface found exactly one disagreement --
endpoint_pools.source, null in production against a required enum, fixed here
at the Zod -- and this is what keeps it at one.
@JSONbored
JSONbored force-pushed the feat/api-response-validation-only branch from 47e1b38 to 859f040 Compare August 10, 2026 12:28
@JSONbored
JSONbored merged commit 884cc9a into main Aug 10, 2026
16 checks passed
@JSONbored
JSONbored deleted the feat/api-response-validation-only branch August 10, 2026 17:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant