Skip to content

linear, fireflies: a request error is Apollo Server's, Linear's introspection, 401 and unknown id are Linear's, and each method, content type and Accept is answered by the vendor's layer - #1027

Merged
nuriyoo merged 37 commits into
mainfrom
graphql-a-request-error-is-apollos
Oct 28, 2026

Conversation

@nuriyoo

@nuriyoo nuriyoo commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

This is part of three issues. It holds blocks 2 to 4 of the Linear one, which is #372, with its block 5 at /graphql and the charset of its block 6. It holds blocks 2 and 4 of the Fireflies one, which is #373, with its block 3 at /graphql and the charset of its block 5, and blocks 1 to 3 of the one about a null variable, which is #918. A refusal Apollo Server or graphql-js makes before a field runs comes back in each vendor's envelope with Apollo Server's code and message, worded by graphql-js where graphql-js made it, in the order both vendors check. Linear answers each one up to the choice of operation whether or not the request carries a credential.

Without a credential, Linear answers a document that selects only __schema and __type, and refuses anything else with Linear's own 401; an unknown id at a root such as issue(id:) meets Linear's Entity not found error in Linear's members. A request to /linear/graphql or /fireflies/graphql by any method uvicorn passes on, under any content type and Accept, is answered by the layer that answers it at the vendor, as the table under Why orders them, but for the requests under Not in this PR.

What changed

The engine (backlot/graphql/engine.py). Engine.resolve_payload and Engine.run answer in the order the module docstring gives. Each refusal is a Refusal, which carries the headers Apollo Server sets on it and whether it is thrown ahead of the pipeline. An empty query sent with its own SHA-256 is parsed, as both vendors parse it. Engine.resolve refuses a GET of a mutation or a subscription with allow: POST, ahead of Linear's credential.

Errors made as an operation runs. A GraphQLError graphql-core raises during the run, such as one for a null variable at a non-null argument, comes back beside data with INTERNAL_SERVER_ERROR, the code Apollo Server sets on an error that carries none. _with_roots adds the root types each vendor names and the SDL leaves out, without fields, so mutation { __typename } answers Mutation.

graphql-js 16's words (backlot/graphql/js_words.py). requote and locations turn graphql-core's messages and locations into graphql-js 16.14.2's. coerce_variable_values refuses variables as graphql-js 16 does, up to the fifty errors execute takes. The module docstring names the kinds of refusal graphql-js 17.0.2 answers otherwise, and what the comparison ran over.

Apollo Server's HTTP layer (backlot/graphql/apollo_http.py). What Apollo Server answers before its request pipeline, read off its source and the two libraries it asks, negotiator 1.0.0 and whatwg-mimetype 4.0.0:

function                    what it is
csrf_refusal                the check that a browser preflighted the request
method_refusal              the refusal of a method but GET and POST
get_content_type_refusal    Apollo Server 5.5's refusal of a GET carrying a type that is not JSON
search_payload              a GET's query string read as URLSearchParams reads it, or its refusal
response_type               the media type Accept takes; NOT_ACCEPTABLE where it takes none
prefers_html, landing_page  the page a browser's GET gets
headers_seen                the request's headers as Node hands them on

A refusal Apollo Server throws ahead of the pipeline takes the media type Accept allows, or JSON; any other answer of Apollo Server's is its 406 where Accept allows none of its types.

Linear (backlot/routers/linear.py). The route picks the operation before it reads the credential, so each refusal up to that point is the same with the key, with no credential and with nope. A variable's refusal and the run wait for the key. _envelope writes Linear's members on each answer, and a field error's in Linear's order, path before locations (_in_linears_order). _variables_read reads a variables string with JSON.parse.

Linear without a credential. A request whose credential does not resolve is served when _introspection_only finds only __schema and __type at the root of the operation it picks. Anything else, __typename included, is the 401. _UNAUTHENTICATED holds Linear's body: the message Authentication required, not authenticated and its seven extensions members, code AUTHENTICATION_ERROR among them.

An id a by-id root holds nothing for (backlot/graphql/linear_resolvers.py). Each by-id root raises _not_found(<type>), Linear's Entity not found: <type> with code INPUT_ERROR and the presentable Could not find referenced <type>.; attachment raises its own words (_ATTACHMENT). A hidden id draws the same error as an absent one, byte for byte (tests/test_acl.py).

Linear's front. The route takes apollo_http.METHODS. The layers in front of Apollo Server answer in the order of the table under Why: _LOAD_BALANCER_501, _preflight, _api_key_as_bearer, then body-parser (_has_body). Each answer carries the media type Express writes (_as_express_sends).

Fireflies (backlot/routers/fireflies.py). _envelope writes each refusal with friendly: false, the code beside the message and empty extensions. _depth_limit is graphql-depth-limit with a maxDepth of 10, run as graphql-js starts validating. Its docstring says what the rule crawls, what it reports and what makes it throw, which Fireflies answers with its 500.

Fireflies' front. In front of Apollo Server come @fastify/cors (_preflight) and Fastify, which reads a body by its type (body.fastify_media_type) and answers a method its route does not take with _not_found. Apollo Server answers a browser's GET with its page, and a CSRF suspect with its 400, before it reads the credential.

Cloudflare (backlot/main.py). TRACE, CONNECT and PURGE under /linear and /fireflies get Cloudflare's own answers (_CLOUDFLARE_METHOD), behind its 400 for Authorization or a conditional header sent twice. The TRACE 405 closes the connection.

NOTICE.md. Apollo Server's MIT notice, for backlot/graphql/apollo_landing_page.html, the page Apollo Server 5.5.1 serves a browser.

Telemetry (backlot/telemetry/errors.py, backlot.graphql.engine._raised). error_code also reads the code Fireflies writes beside the message. A GraphQL refusal is counted by the code its envelope carries (Apollo Server's, or Linear's own INVALID_INPUT, INPUT_ERROR and AUTHENTICATION_ERROR), not by a slug read from its message. The slug unsupported_filter counts the Linear resolvers' own refusals that start unsupported, of a filter, a comparator or a sort key.

Fireflies' auth_failed 500, on any method, is counted with the backlot.error_class auth (_AUTH), not as Backlot's own failure. The failures among the answers that newly reach the classifier, the last seven as test_a_call_is_named_by_the_baseline holds them:

answer                                       error code              error_class
Fireflies' 500 when graphql-depth-limit     INTERNAL_SERVER_ERROR   client, by _ANSWERED_5XX
  throws
Fireflies' 200 refusing a subscription       INTERNAL_SERVER_ERROR   client: _raised reads graphql-core's error on the operation as a refusal
Fireflies GET, Bearer nope                   auth_failed             auth
Fireflies PROPFIND                           not_found               client
Fireflies OPTIONS, unpreflighted             (none)                  client
Linear PUT                                   BAD_REQUEST             client
Linear POST, Accept: text/html               BAD_REQUEST             client
Linear BIND                                  (none)                  client
Linear TRACE                                 (none)                  client

Each answer carries its own status (Result.status). The engine answers no refusal in express-graphql's words (POST body sent invalid JSON., Must provide query string.), since neither vendor does.

Docs. docs/supported-sources.md says each endpoint takes a GET, and when Apollo Server serves one.

Tests. test_linear_reads_a_credential_as_linear_does and test_linear_orders_the_body_and_the_credential hold the 401's bytes for each credential that is not one, and the second, in its _UNKNOWN_IDS rows, the not-found's bytes for each by-id root with the key, issue(id: "NOPE-1") among them.

test_linear_answers_each_method_type_and_accept_as_linear_does and test_fireflies_answers_each_method_type_and_accept_as_fireflies_does hold 163 and 175 requests as each vendor answered them, and test_cloudflare_answers_ahead_of_linear_and_fireflies Cloudflare's answers on the other paths. Five tables in tests/test_graphql.py hold what each library answered under Node.

Not in this PR. The drafts stacked on this one, #1058 to #1063, hold the rest of the three issues. A variable named $__proto__, which Linear answers as if it had no value, is #1113. The members of an Argument Validation Error, validationErrors among them, are #370's. Which of a by-id root's not-found and a later root field's refused arguments Linear answers is #1067. A request uvicorn's parser refuses gets uvicorn's 400 before the app is called, where each front answers its own, which is #1072.

the paths other than /graphql                                   the rest of #372 block 5, #373 block 3   #1058
the headers, and the newline each Apollo body ends with         #372 block 6, #373 block 5               #1061, #1062
which of a resolver's error and a later root field's refused    #918 block 4                             #1063
  argument is answered

A suggestion graphql-js makes from a type Backlot's SDL does not declare (Did you mean "Node"? on Linear, "Role" on Fireflies) needs no issue, because the SDL declares the types Backlot serves. Introspection answers that SDL, with or without Linear's key, so mutationType is null where Linear names Mutation. That needs none either: test_linear_introspection_reports_the_served_schema and test_fireflies_declares_no_mutations hold it, so a client generating calls from it is offered no write.

Each of four answers that differ needs no issue, for the reason beside it:

request                               the vendor                     here                    why no issue
a GET or a HEAD carrying a body       Linear: Cloudflare's 500       the body left unread    Linear's front fails on it
                                        "error code: 1101", a 400
Access-Control-Request-Headers ` , `  Linear: an empty list          `,`                     a list of no header names
a name in it with a byte past ASCII   Linear drops one at an end,    the name trimmed of     a header name is ASCII
                                        turns U+0085 into U+00A0       spaces and tabs
a persisted query's hash alone,       Fireflies served it once in    PersistedQueryNotFound  the protocol's answer for a
  its query sent before                 16, as this answers the 15                             query to be sent again

Why this is what the real API does

Measured. api.linear.app with a personal API key and api.fireflies.ai with a bearer key on 2026-10-25 (UTC), one request per row with its own cache-buster. The requests sent each of Apollo Server's body checks alone and two at a time, persisted queries, and a refusal at each stage. They also sent errors made as the operation ran, fragments and the extra root types, and every body the tables hold that reaches Apollo Server, with the key, with no credential and with nope. Among them:

request                                         answer
{} or an empty body                             POST body missing, invalid Content-Type, or JSON object has no keys.
Linear, variables "{}", " {} " or "null"        served: Linear reads the string with JSON.parse
Linear, variables "{", "" or "NaN"              `variables` ... not a recursively JSON-encoded string.
Linear, variables "[1]" or "5"                  `variables` in a POST body must be an object if provided.
Linear, operationName "Z"                       operation does not exist, The operation 'Z' does not exist on the query.
a Fireflies refusal                             {"friendly":false,"message":…,"code":…,"extensions":{}}

Replayed over backlot.serve() at 4624cbdb, comparing the status and the body without its trailing newline:

requests                                        sent   same   the rest
the key only                                     256    250   6  a suggestion from a type the SDL does not declare
the key, no credential and nope                  231    213   18 Linear's 401 body (#372 block 3)
the key only, 2026-10-26 (below)                  42     39   3  a field error's presentable message (#372 block 4)

The earlier dates the code keeps, 2026-10-18 and 2026-10-19 for Linear's Bearer 400 and Express, and 2026-10-21 and 2026-10-22 for the members of a field error, are measurements this branch moves rather than makes.

A persisted query and an operation's name. Both vendors on 2026-10-28 (UTC), each request with its own cache-buster, answered as backlot.serve() does at 722d3ce8:

request                                              Linear                    Fireflies, with the key
query "" with the SHA-256 of ""                      400 Syntax Error:         400 Syntax Error:
                                                       Unexpected <EOF>.         Unexpected <EOF>.
query "" with no extension                           400 must contain a        the same
                                                       non-empty `query` …
query __proto__ { __typename }                       200                       500 Internal server error
query A { __typename } query __proto__ { … },        200                       500
  operationName A
query constructor, or __defineGetter__               200                       200
a spread of a fragment named __proto__               200                       200

Linear answered the first two rows the same with no credential. graphql-depth-limit 1.1.0, run locally under graphql-js 16.14.2, throws on the two __proto__ operation rows and passes the rest.

graphql-depth-limit and the locations past U+FFFF. api.fireflies.ai and api.linear.app on 2026-10-26 (UTC), the same way:

request                                                answer
Fireflies, { ...X } under __schema, __type or          400, the validation's errors alone: a field named __*
  __typename                                             is not crawled
Fireflies, { ...X } { __typename }, or query A         400: the later operation of a name is the one crawled
  { ...X } query A { __typename }
Fireflies, query B { ...X } query A { __typename }     500
Fireflies, an undefined spread ten fields deep         500
  eleven fields deep                                   400, '' exceeds maximum operation depth of 10 first
Fireflies, 99, 100, 101 fields twelve deep             400 with 99 depth errors and the unknown field; 400 with
                                                         100 and Too many validation errors; 500
Fireflies, 50 and 51 such fields in two operations     500
Linear, Cannot query field after two U+1F600           column 61, the code points' count plus two
Linear, Entity not found after one U+1F600             column 66, the code points' count plus one
the 51st refusal of a variable                         Fireflies: Too many errors processing variables, ...,
                                                         INTERNAL_SERVER_ERROR; Linear: Internal server error,
                                                         no http member

Apollo Server and graphql-js, run locally on 2026-10-25.

compared                                          with                        sent     agreed
Apollo Server 5.5.1 behind Express 5 and cors,    Linear's measured answers   each     all but where Linear writes its own
  over Linear's SDL                                                                      (INVALID_INPUT, operation does not
                                                                                         exist, Internal server error, a variables
                                                                                         string read with JSON.parse), keeps no
                                                                                         persisted query, and an OPTIONS without Origin
Apollo Server 4.13.0, as it is set up by default  both vendors' refusal of    each     no: 200 where both answered 400
                                                    a variable
js_words                                          graphql-js 16.14.2,         12,643   all, messages and locations
                                                    Node 24.18.0, both SDLs
js_words                                          the same, a schema of       852      all
                                                    every input kind
coerce_variable_values                            graphql-js's coercion       16,542   all
graphql-core's own coercion                       graphql-js's                4,770    takes each value graphql-js takes

The code names neither Apollo Server major, as 4.13.0 configured otherwise cannot be ruled out.

The credential. The order of the checks is in body_refusal's docstring, and every pair the tables hold was sent with both members wrong. Each refusal measured up to the choice of operation, on 2026-10-25 (UTC):

sent                                   Linear                 Fireflies
with the key                           the refusal            the refusal
with no credential, or nope            the same refusal       its auth_failed 500
a document missing a variable,         the 401                its auth_failed 500
  no credential

graphql-js. Linear's refusal of an array where an input object goes is graphql-js 16.14.2's, where 16.9.0 takes each index of the array for a field's name. test_a_refusal_is_in_graphql_js_words holds 42 rows, 38 of them from the 852 and three from graphql-js 16.14.2 run on 2026-10-26.

Introspection, the 401 and an unknown id. api.linear.app on 2026-10-25 (UTC), one request per row with its own cache-buster:

request, no credential                                         answer
{ __schema { queryType { name } } }, aliased, as a named query    200, served
__schema in a named or an inline fragment, or under @skip(if: false)   200, served
{ __schema … } __type(name: "Issue") …                          200, served
query A { __schema … } query B { viewer … }, A or B picked        200 for A, the 401 for B
{ __typename }, or beside __schema, or in a fragment             the 401
{ __schema … viewer @skip(if: true) { id } }                     the 401
query($b: Boolean!) { __schema @include(if: $b) … } with no $b   400, the variable's refusal
{ viewer { id } }, with no Authorization, Bearer nope, nope,     the 401
  Bearer, or Basic Zm9vOmJhcg==

With the key:

request                                                   answer
an unknown UUID or nope at issue, team, user,             Entity not found: <type>, the not-found's members in the
  workflowState, project, issueLabel, cycle, release,       order message, path, locations, extensions
  attachment, issueRelation and teamMembership, and
  NOPE-1 at issue
issues with both sort and orderBy                         Cannot use both sort and orderBy options, in that order
issues(filter: {id: {eq: "!!"}})                          Argument Validation Error, in that order

Replayed over backlot.serve() at 7c8e02f1, comparing the status and the body without its trailing newline:

requests                                   sent   same   the rest
introspection and the 401                    52     46   6  types the SDL lists, a corpus viewer id, Linear's root types,
                                                            an Argument Validation Error's members (#370)
an id a by-id root holds nothing for         26     22   4  document and comment, roots Backlot's SDL does not declare

Methods, content types and Accept. api.linear.app and api.fireflies.ai on 2026-10-25 (UTC), with the key, no credential and nope, and on Linear an API key under Bearer, one request per row with its own cache-buster:

sent                                                 Linear   Fireflies
requests                                             454      442
methods: each uvicorn passes on                      35       35
Content-Type values, across both                     69
Accept values, across both                           41

Among them, and on 2026-10-26 one connection probe per answer, the empty Origins and the plain text holding 0xE9:

request                                              answer
twelve GETs preferring text/html                     Fireflies: Apollo Server's page, 3522 bytes but for its nonce
a form holding query=%7B%20__typename%20%7D          Fireflies: served, beside the preflight header
Accept: application/json; callbackSpec=1.0           Linear: application/json; callbackspec=1.0; charset=utf-8
                                                     Fireflies: application/json; callbackSpec=1.0; charset=utf-8
TRACE, then a GET on the same connection             both: Connection: close on the 405, no answer to the GET
CONNECT or PURGE, then a GET on the same connection  both: Connection: keep-alive on the 400, the GET answered
OPTIONS, Origin empty or blank                       Linear: CSRF 400, or the 405 with a JSON type
                                                     Fireflies: 400 Invalid Preflight Request, and so for an
                                                       empty Access-Control-Request-Method
text/plain holding 0xE9, by POST or PUT              Fireflies: 400 Request body size did not match
                                                       Content-Length; served as a form; 413 past the limit

On 2026-10-28 (UTC), each request on a connection of its own with its own cache-buster:

request, on /graphql, /graphql/, /nosuch and /       answer, on both
TRACE                                                405, text/html, 155 bytes, Connection: close
CONNECT                                              400, text/plain; charset=UTF-8, 17 bytes, keep-alive
PURGE                                                400, text/html; charset=UTF-8, 155 bytes, keep-alive
TRACE, CONNECT or PURGE with Authorization,          400, the page, text/html, 155 bytes, Connection: close
  If-None-Match or If-Modified-Since sent twice
GET ?query= with the SHA-256 of "" as a persisted    400, Syntax Error: Unexpected <EOF>.
  query, with the key and apollo-require-preflight
GET ?query= alone                                    400, must contain a non-empty `query` …
GET with the persisted query's hash alone            200, PersistedQueryNotFound

backlot.serve() at 270eaf3c answers each with the vendor's status, media type and body. The head holds the Linear, Fireflies and GraphQL code of 270eaf3c unchanged but for js_words' docstring.

The layers answered in this order:

layer               Linear                                              Fireflies
Cloudflare          TRACE 405, CONNECT 400 "error code: 1045", PURGE 400, each behind its 400 for Authorization or a
                    conditional header sent twice
load balancer       empty 501: BIND LINK M-SEARCH MKCALENDAR REBIND      (none)
                    SOURCE UNBIND UNLINK
CORS                cors: OPTIONS with Origin, 204                       @fastify/cors: OPTIONS, 204 with Origin and
                                                                         Access-Control-Request-Method, else 400
credential first    an API key under Bearer, on all but OPTIONS          (none)
body reader         body-parser: a JSON type, on all but GET and HEAD    Fastify: JSON, text, form; 415 for any other on
                                                                         POST; a 404 for PUT, DELETE, PATCH after the body
Apollo Server       CSRF, 405, GET query string, pipeline,               landing page, CSRF, credential, 405 for HEAD,
                    405 for a GET not a query, credential, run, 406      GET 415, GET query string, pipeline, run, 406

Which releases. Fireflies' landing page names @apollo/server@5.5.1, and its 415 for a GET's content type came with Apollo Server 5.5.0; Linear answers no such 415. Run locally:

compared                                     with                             answer
13 Fastify 5 releases under Apollo Server    Fireflies' requests              5.7.4 to 5.10.0 match each but four Cloudflare
  5.5.1                                                                         answers and one persisted query; 5.5.0 refuses
                                                                                application/json\t;, 5.7.0 serves
                                                                                application/json , and 5.11.3 on read a QUERY's body
type-is 2.0.1 and 2.1.0                      Linear's body-parser on 66       2.0.1 on all 66; 2.1.0 unlike it on eight
                                               content types
negotiator 1.0.0 and 1.1.0                   each vendor on the nine Accept   1.0.0
                                               values where the two part

Replayed over backlot.serve() at 270eaf3c, each request to /graphql itself with a method uvicorn passes on (the other paths are the next draft's, which is #1058), comparing the status, the media type and the body without its trailing newline:

vendor      sent   same   the rest
Linear      414    410    4  a control character in a header, a GET or a HEAD carrying a body
Fireflies   405    403    2  a control character in a header, a persisted query's hash alone

Verification

  • Tests — pytest on uv sync --all-extras --locked plus uv pip install --no-deps "llama-index-readers-hubspot<0.6", as CI installs: 8708 passed, 1 skipped.
  • Optional surfaces that ran rather than skipped — every one but the Atlassian MCP test in tests/test_mcp.py, which needs Docker's Linux containers and found no daemon here.
  • Lint — ruff check . && ruff format --check .: clean.

At 1bad7ec9, on main at d0934c22.

uv sync --all-extras --locked
uv pip install --no-deps "llama-index-readers-hubspot<0.6"
uv run --no-sync pytest -rs                 8708 passed, 1 skipped in 58.59s
ruff check . && ruff format --check .       All checks passed! / 193 files already formatted
python scripts/prose_check.py origin/main   exit 0
mutations of each line this branch adds     991, 945 caught
  generated, by hand                        966, 25
  left: differing past ASCII only, none     2, 44

Each generated mutation changes one decision on a line this branch adds over main; those by hand cover the depth rule, the cap on a variable's errors, the UTF-16 column, each call of _not_found, the introspection a request with no key is served, the preflights and the body's length check. Of those left, the trims of a preflight's header names (the table under Not in this PR.) and of an Accept parameter differ only past ASCII, and the rest cannot change an answer. The run is on the tree before main's last two commits were merged in, which change GitHub's and Slack's baselines and GitHub's router, none of the files it mutates and none of its test files:

swaps of two checks that cannot both hold, and of the two halves of a pure `and`
comparisons widened to methods an earlier check refuses or no caller passes
Cloudflare's three methods in the route's list, which backlot.main answers first
negotiator's plain multipart/mixed, which the two typed ones always tie or beat
guards for the nodes every error of a run has
the depth rule's errors dropped from the refusal test (Fireflies' SDL nests objects four deep)
a type alias's members, two of them leaving the module unimportable
the not-found of _issue_by_id, which no request reaches
one line this branch did not write
  • A test fails without this change — test_linear_orders_the_body_and_the_credential, test_fireflies_orders_the_body_and_the_credential, test_linear_answers_each_method_type_and_accept_as_linear_does, test_cloudflare_answers_ahead_of_linear_and_fireflies, test_each_graphql_error_is_counted_by_the_code_backlot_sends.
  • A new endpoint serving corpus content is ACL-scoped, proved for both an admin and a scoped token — test_linear_by_id_roots_do_not_leak_entities_off_hidden_issues, test_fireflies_transcripts_list_hides_denied_meetings.
  • Comments state what was measured, not the history of the fix

Both vendors run Apollo Server 4 over graphql-js. The engine now answers
a request the way that pair does: Apollo Server's checks of the body and
of a persisted query, then graphql-js's parse, validation, choice of
operation and coercion of variables, each refusal with Apollo Server's
code and graphql-js's wording (backlot.graphql.js_words). Each router
writes the refusals in its vendor's envelope, and Linear answers them
up to the choice of operation before it reads the credential.

The telemetry classifier reads Fireflies' code beside the message, and
the GraphQL message slugs those codes replace are gone.
…at JSON cannot reach

A table of documents over a schema of every input kind holds graphql-js
16.14.2's answers under Node, picked so that each change to js_words a
row can tell apart has one. The vendor tables gain the rows Linear and
Fireflies answered for the rest: an error's place after a line
separator, a persisted query's version, the unreachable fragments, and
the body checks two at a time.

js_words drops the branches no request reaches: a circular value, a
variable whose type validation already refused, a scalar's GraphQLError.
…looked up

Both vendors answered a hash sent with `query` false or 0 as with no
query at all, PersistedQueryNotFound at 200, and hashed an empty string.
…ot Backlot's failure

Fireflies has no subscription type, and graphql-core refuses a
subscription with the operation as the error's node. That is the
request's fault, so the call is counted `client`, as the 200 Fireflies
answers with says.
…iteral, and a refusal is read off the field error's own nodes
@nuriyoo nuriyoo added fidelity The vendor API defines the right answer; closing this needs a measurement against it telemetry Usage telemetry: how a call is named, counted and exported, and server changes that alter it labels Oct 25, 2026
@nuriyoo nuriyoo self-assigned this Oct 25, 2026
…ounts as graphql-js counts

Fireflies runs graphql-depth-limit (maxDepth 10) as graphql-js starts
validating: its depth errors come ahead of graphql-js's own and count
toward the hundred validate takes, and it throws, which Apollo Server
answers with its 500, on a spread of an undefined fragment, a
field-less spread cycle and a 101st report. That replaces the walk
for undefined fragments, which answered the 500 under __schema,
__type and __typename and for operations a later one replaces.

A column counts UTF-16 code units, and each error the run makes,
a resolver's too, is located as graphql-js locates it. The refusal
after fifty refused variables keeps INTERNAL_SERVER_ERROR, which
Linear masks as its Internal server error.
…or-is-apollos

# Conflicts:
#	tests/test_fireflies.py
#	tests/test_linear.py
…or-is-apollos

# Conflicts:
#	backlot/routers/fireflies.py
@nuriyoo
nuriyoo marked this pull request as ready for review October 27, 2026 15:20
@nuriyoo
nuriyoo requested a review from khj809 October 27, 2026 15:20

@khj809 khj809 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 4624cbdb, split into behaviour (re-measured live) and prose. Verdict: changes requested. There are two behaviour blocks and two prose blocks, and I've also added a note on how js_words.py is scoped to graphql-js 16.

I sent 55 requests to api.linear.app and api.fireflies.ai on 2026-10-28 (UTC), each with its own cache-buster, and sent the same requests to backlot.serve() at the head. They matched except for the two blocks below and the 401 body this PR already declares (#372 block 3). The refusals Linear now answers without a credential come from resolve_payload over the body and the SDL, and no store read is involved. When the credential check is moved ahead of resolve_payload, test_linear_orders_the_body_and_the_credential fails.

Behaviour

[block] backlot/graphql/engine.py:427 — an empty query whose persisted-query hash matches is refused as having no query.

{"query":"","extensions":{"persistedQuery":{"version":1,"sha256Hash":"e3b0c442…b855"}}}   (sha256 of "")

api.linear.app (key, and no credential)   400  Syntax Error: Unexpected <EOF>.  line 1 col 1  GRAPHQL_PARSE_FAILED
api.fireflies.ai                          400  Syntax Error: Unexpected <EOF>.  line 1 col 1  GRAPHQL_PARSE_FAILED
Backlot, Linear                           400  GraphQL operations must contain a non-empty `query` or a `persistedQuery` extension.  INVALID_INPUT
Backlot, Fireflies                        400  same message  BAD_REQUEST

Apollo Server only refuses a body for having no query when there is no persistedQuery extension. When there is one, a string query whose hash matches goes on to be parsed. _persisted(query="") in tests/test_graphql.py:953 covers only a hash that does not match. The fix is to apply NO_QUERY only when there is no truthy persistedQuery, and to add the matching-hash row to test_apollo_server_reads_a_body_in_its_order and to both vendors' order tables.

[block] backlot/routers/fireflies.py:228 — an operation named __proto__. graphql-depth-limit keeps operations in a plain {} and walks them with for…in, so on Fireflies every document with an operation named __proto__ gets the 500.

query __proto__ { __typename }          api.fireflies.ai 500 {"friendly":false,"message":"Internal server error","code":"INTERNAL_SERVER_ERROR","extensions":{}}
                                        Backlot          200 {"data":{"__typename":"Query"}}
query __proto__ { …12 fields deep… }    api.fireflies.ai the same 500
                                        Backlot          400 '__proto__' exceeds maximum operation depth of 10, plus a validation error
query constructor { __typename }        200 on both

The fix is for _depth_limit to return the 500 refusal when any operation is named __proto__, with a row added to test_fireflies_orders_the_body_and_the_credential. This case is narrow, so I'd fix the empty-query one first.

Prose

[block] backlot/graphql/engine.py:111 and the module docstring at engine.py:12-13 — "message in graphql-js's words". Several refusals are not in graphql-js's words. body_refusal and persisted_query_refusal use Apollo Server's own messages (Operation batching disabled., PersistedQueryNotFound, Unsupported persisted query version), and _depth_limit uses graphql-depth-limit's. Suggested wording: "message as Apollo Server writes it, in graphql-js's words where graphql-js made the refusal".

[block] backlot/graphql/engine.py:475 — "graphql-core's ExecutionResult.formatted writes "data" first; …". run no longer calls result.formatted; it builds payload from each error.formatted. That clause only contrasts with the deleted line. The fix is to start the comment at "api.fireflies.ai and api.linear.app open a response carrying a field error with "errors" (2026-10-21), …".

[note] persisted_query_refusal docstring (engine.py:262-269). The docstring describes PersistedQueryNotFound first and the version check later, but the code checks the version first: a hash with no query and version 2 gets Unsupported persisted query version. The docstring should follow the code's order.

[note] backlot/routers/fireflies.py:199-201 — _MAX_ERRORS = 100. This repeats _MAX_VALIDATION_ERRORS = 100 in engine.py:104-106. The router could use the engine's constant and keep only the "a 101st report from a rule throws past" clause.

[note] PR body. Six "What changed" paragraphs run to 476–780 characters. The Fireflies paragraph restates _depth_limit's docstring and the Linear paragraph restates _envelope's. "The credential." and the mutation paragraph under Verification put measurements in prose; each would read better as a small table.

js_words.py is graphql-js 16's words, and its head should say so

I ran graphql-js 16.14.2 and 17.0.2 (npm latest) side by side on 19 cases.

same in 16.14.2 and 17.0.2                 syntax errors, " quoting, directive location names (QUERY), unknown-field and enum
                                           suggestions, leaf selection, inspect truncation, the 50-error cap on variables
reworded in 17                             every refusal of a variable, and the missing-argument validation message:
  var wrong type      16  Variable "$v" got invalid value "abc"; Int cannot represent non-integer value: "abc"
                      17  Variable "$v" has invalid value: Int cannot represent non-integer value: "abc"
  var null            16  Variable "$v" of non-null type "Int!" must not be null.
                      17  Variable "$v" has invalid value: Expected value of non-null type "Int!" not to be null.
  var missing         16  Variable "$v" of required type "Int!" was not provided.
                      17  Variable "$v" has invalid value: Expected a value of non-null type "Int!" to be provided.
  list item path      16  … got invalid value null at "i.b[0]"; …
                      17  … has invalid value at .b[0]: …
  missing argument    16  Field "f" argument "x" of type "Int!" is required, but it was not provided.
                      17  Argument "Query.f(x:)" of type "Int!" is required, but it was not provided.

So coerce_variable_values is entirely graphql-js 16, and so is requote's rewording of the missing-argument message. If a vendor moves to 17, nothing raises; the answers just diverge. Both vendors answered in 16.14.2's words on 2026-10-25. Upgrading graphql-core to 3.3 (the port of 17) would change requote's input, but pyproject.toml:43 already pins <3.3 and states why.

The docstring names 16.14.2 only partway into a sentence about the measurement date, and its summary line names no version. I'd put the scope at the head and leave the pin's reason in pyproject:

"""graphql-js 16's words for the errors graphql-core 3.2 makes with its own.

Linear and Fireflies answered in graphql-js 16.14.2's words on 2026-10-25 (UTC). graphql-js 17
rewords each refusal of a variable and of a missing argument (17.0.2, run on 2026-10-28), so a
vendor's upgrade shows here as a divergence, not an error.

…mpty, and Fireflies refuses an operation named __proto__

Apollo Server refuses a body for having no query only when it sends no
persistedQuery extension; with one, a string query whose hash matches,
"" included, goes on to be parsed. Both vendors answered "" with its hash
with graphql-js's Syntax Error on 2026-10-28 (UTC).

graphql-depth-limit keeps the operations in a plain object, so an operation
named __proto__ becomes its prototype and its for...in throws. Fireflies
answers any document holding one with its 500.

The engine's MAX_VALIDATION_ERRORS is now the one constant the Fireflies
depth rule reads. Docstrings say a refusal's message is Apollo Server's,
in graphql-js's words where graphql-js made it, and js_words names
graphql-js 16 as its scope.
@nuriyoo nuriyoo changed the title linear, fireflies: a request error is Apollo Server's in graphql-js's words, and Linear answers one before the credential linear, fireflies: a request error is Apollo Server's, and Linear answers one before the credential Oct 28, 2026
@nuriyoo

nuriyoo commented Oct 28, 2026

Copy link
Copy Markdown
Contributor Author

Each point of the review, at 722d3ce8:

  • The empty query with a matching hash — 326fc667. resolve_payload refuses a body for having no query only when it sends no persistedQuery extension (_persisted_query, which persisted_query_refusal reads too), so "" with its hash is parsed. Rows: pq matching an empty query in test_apollo_server_reads_a_body_in_its_order, and a persisted query's hash matching an empty query in both vendors' order tables. Both vendors answered it, by POST and by GET, with the Syntax Error on 2026-10-28, Linear with and without a credential.
  • __proto__ — 326fc667. _depth_limit throws for a document holding an operation of that name, and its docstring says why (graphql-depth-limit's plain object takes it as its prototype; graphql-depth-limit 1.1.0 run locally throws on it). Rows in test_fireflies_orders_the_body_and_the_credential: the operation alone, beside the one chosen, and three served controls Fireflies answered the same day (constructor, __defineGetter__, a fragment named __proto__); Linear's table has query __proto__ served.
  • "in graphql-js's words" — 326fc667 and ce5cc228: Refusal has your wording, and the module docstring names a refusal's code and message and leaves the words to Refusal, so the phrase has one home. The PR title says only that a request error is Apollo Server's.
  • The ExecutionResult.formatted clause — 326fc667; the comment starts at the measurement.
  • persisted_query_refusal's order — 326fc667: the version first, then the missing query, then the hash, as the code runs them.
  • _MAX_ERRORS — 326fc667: the router reads engine.MAX_VALIDATION_ERRORS, and its docstring keeps what a report past it does.
  • js_words's scope — 326fc667 and 8b7d08ac: the summary line names graphql-js 16 and graphql-core 3.2, and the first paragraph gives the 16.14.2 measurement and the five refusals 17.0.2 words otherwise (I ran the two side by side and got your four variable cases and the missing argument).
  • The body — rewritten: What changed points at the docstrings, the credential and the 2026-10-28 cases are tables, and the mutation count is a row of the command table.

A variable named $__proto__ is a gap of the same kind at Linear's end, though not graphql-js's (it gathers variables in Object.create(null), and Fireflies served one): filed as #1113.

@khj809

khj809 commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Rechecked at cf7a758c; every point of the review is addressed. I sent the fixed cases to the live APIs and to backlot.serve() on 2026-10-28 (UTC), and all 14 matched byte for byte. On Linear these were the empty query with its matching hash (with the key and with no credential) and query __proto__. On Fireflies they were the empty query with its matching hash, plus __proto__ alone, twelve fields deep and beside the chosen operation, with constructor, __defineGetter__, toString, hasOwnProperty and a fragment named __proto__ as controls.

One correction is left, and it comes from my own review: its table showed four variable cases as examples, but they weren't the whole set. js_words.py's docstring now says graphql-js 17.0.2 "words five refusals otherwise". When I ran 16.14.2 and 17.0.2 side by side, ten of the cases differed: every variable refusal I sent, plus the missing argument.

reworded in 17.0.2                      listed in the docstring
a variable of the wrong type            yes
a null non-null variable                yes
a missing non-null variable             yes
a list item its type refuses            yes
an array sent for an input object       no  (16: Expected type "I" to be an object. / 17: Expected value of type "I" to be an object, found: [1].)
an input object missing a field         no  (17: Expected value of type "I" to include required field "a", found: {}.)
an input object with an unknown field   no  (17: Expected value of type "I" not to include unknown field "zz", found: { a: 1, zz: 2 }.)
a value not in an enum                  no  (16: got invalid value "Z"; Value "Z" … / 17: has invalid value: Value "Z" …)
an inspect-printed non-scalar value     yes, as the wrong type
a missing argument                      yes

Every 17 variable refusal opens with Variable "$v" has invalid value where 16 opens with got invalid value, or with of non-null type / of required type. A sentence that doesn't need a count would avoid this: "graphql-js 17.0.2 words each refusal of a variable, and of a missing argument, otherwise (run on 2026-10-28 (UTC))."

Apart from that sentence, this is good to go.

…herwise

The five refusals the docstring listed were a sample. Run over documents for
each of graphql-js 16.14.2's validation rules, lexer and parser errors and
value coercions, 17.0.2 also words otherwise the refusal of an argument's
value as the field runs, a null literal for a non-null type, an input object
literal of the wrong shape, a custom scalar's literal refusal, and the syntax
errors at `.5` and `..`; it refuses a subscription to a schema without one as
it validates, adds refusals for a subscription root's @skip/@include and for
@defer/@stream, and orders an unused variable's refusal first.
@nuriyoo

nuriyoo commented Oct 28, 2026

Copy link
Copy Markdown
Contributor Author

The sentence is fixed in 62f63518, and it no longer counts anything. I didn't use the suggested sentence as it stood, because the set is wider than the variable refusals and the missing argument.

On 2026-10-28 (UTC) I ran graphql-js 16.14.2 and 17.0.2 side by side on 162 documents. Each of 16.14.2's 27 validation rules fires on at least one of them, and they also cover its lexer and parser errors and each branch of its value coercion. The schemas match the two here in shape: a field-less Subscription root for Linear, and no Subscription root for Fireflies. 59 documents got a different answer. Besides every variable refusal you sent and the missing argument (which differs for a directive as well as a field), these differ, one example each:

a null literal for a non-null type (argument, list item, variable default)
  16  Expected value of type "Int!", found null.
  17  Expected value of non-null type "Int!" not to be null.
an input object literal without a required field (an unknown field and a non-object differ too)
  16  Field "I.a" of required type "Int!" was not provided.
  17  Expected value of type "I" to include required field "a", found: { b: "x" }.
a literal a custom scalar refuses with a plain Error
  16  Expected value of type "D", found 1; Unable to parse literal value of kind 'IntValue'. D supports only 'StringValue' ones
  17  Expected value of type "D", but encountered error "Unable to parse literal value of kind 'IntValue'. D supports only 'StringValue' ones"; found: 1.
a null variable with a default, given to a non-null argument (refused as the field runs)
  16  Argument "a" of non-null type "Int!" must not be null.
  17  Argument "Query.f(a:)" has invalid value: Expected variable "$v" provided to non-null type "Int!" not to be null.
{ g(fl: .5) }
  16  Syntax Error: Unexpected character: ".".
  17  Syntax Error: Invalid number, expected digit before ".", did you mean "0.5"?
{ t { ..x } }
  16  Syntax Error: Unexpected character: ".".
  17  Syntax Error: Unexpected "..", did you mean "..."?
subscription { __typename }, on a schema with no Subscription root
  16  Schema is not configured to execute subscription operation.            (as it executes)
  17  The subscription operation is not supported by the schema.             (as it validates)
subscription { __typename @include(if: true) }
  16  Anonymous Subscription must not select an introspection top level field.
  17  Anonymous Subscription must not use `@skip` or `@include` directives in the top level selection.
{ nn @stream }, with neither directive in the schema
  16  Unknown directive "@stream".
  17  Unknown directive "@stream".  Directive "@stream" cannot be used on non-list field "Query.nn".
query($v: Zz) { nn }
  16  Unknown type "Zz".  Variable "$v" is never used.
  17  Variable "$v" is never used.  Unknown type "Zz".

The rest answered the same: the other validation rules, every other lexer and parser error, a literal of the wrong built-in scalar or enum, and a custom scalar's literal refusal thrown as a GraphQLError. A OneOf input and the token limit also differ, but this module words neither of them. Neither SDL uses @oneOf, and the engine sets no token limit.

The docstring now names each of these kinds and says what the run covered.

…near's, and an unknown id is Linear's INPUT_ERROR (#1028)
@nuriyoo nuriyoo changed the title linear, fireflies: a request error is Apollo Server's, and Linear answers one before the credential linear, fireflies: a request error is Apollo Server's, Linear's introspection, 401 and unknown id are Linear's, and each method, content type and Accept is answered by the vendor's layer Oct 28, 2026
@nuriyoo

nuriyoo commented Oct 28, 2026

Copy link
Copy Markdown
Contributor Author

The pull request stacked on this one, which is #1028, and the one stacked on that, which is #1029, are squash-merged into this branch as fbb2247e and 8aa57901, each after its approval. The tree at 8aa57901 is the tree the three heads merge to, and the full suite passes on it (8701 passed, 1 skipped). The title and body now cover all three, and the mutation run in the body is over the whole diff against main.

@nuriyoo nuriyoo added the docs The repo's own documentation: README, docs/, CONTRIBUTING label Oct 28, 2026

@khj809 khj809 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked at 1bad7ec9, and approving. The ten examples in your reply match 16.14.2 and 17.0.2 exactly when I run them, and the docstring now lists each kind of refusal that changes and no longer counts them. The #1028 and #1029 squashes carry the heads I approved. Apart from js_words.py and the merges from main, nothing changed in the routers, graphql/ or the tests, and the linear.py conflict resolves to the approved version. CI is green on 3.11–3.15.

@nuriyoo
nuriyoo merged commit 96c2ceb into main Oct 28, 2026
9 checks passed
@nuriyoo
nuriyoo deleted the graphql-a-request-error-is-apollos branch October 28, 2026 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs The repo's own documentation: README, docs/, CONTRIBUTING fidelity The vendor API defines the right answer; closing this needs a measurement against it telemetry Usage telemetry: how a call is named, counted and exported, and server changes that alter it

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants