Repository navigation
linear: a body sent as br, utf-7 or utf-7-imap is decoded as Linear's body-parser decodes it - #1108
Conversation
…nd Brotli tables gain the rows their mutations asked for
… and point at LINEAR_LIMIT
…linear-a-utf-7-body-is-read-as-iconv-lite-reads-it
…linear-a-utf-7-body-is-read-as-iconv-lite-reads-it
…linear-a-utf-7-body-is-read-as-iconv-lite-reads-it
…linear-a-utf-7-body-is-read-as-iconv-lite-reads-it
…linear-a-utf-7-body-is-read-as-iconv-lite-reads-it
…t what end does; Notion reads utf-7 as Linear does
…linear-a-utf-7-body-is-read-as-iconv-lite-reads-it
…linear-a-utf-7-body-is-read-as-iconv-lite-reads-it
…f a surrogate pair echoed in a variable's error
…linear-a-utf-7-body-is-read-as-iconv-lite-reads-it
…linear-a-utf-7-body-is-read-as-iconv-lite-reads-it
…linear-a-utf-7-body-is-read-as-iconv-lite-reads-it
khj809
left a comment
There was a problem hiding this comment.
Reviewed for behaviour (live calls to api.linear.app and api.notion.com on 2026-10-29, the PR's server under uvicorn) and for prose.
Behaviour: pass
All 41 br, BR, utf-7 and utf-7-imap rows of _LINEAR_READS, plus 32 further malformed or combined bodies (br garbage, a br stream past the limit and cut short, a valid start followed by garbage, a second truncated stream, br around utf-7 and utf-7-imap; utf-7 runs of +AHs- and +AAA-, a high byte or = in a run, a lone low surrogate, gzip around a run, a mark at the end; utf-7-imap runs closed by a space, high bytes in and out of a run, a lone high surrogate, a mark in a run, & at the end; charset names UTF-7, UTF-7-IMAP, utf7) came back from Backlot with the same status and body bytes as Linear, 73 of 73. _utf7 agreed with iconv-lite 0.7.3's decoder on 150,000 generated byte strings in both charsets. Nine utf-7 rows against Notion's POST /v1/search matched too. On the base with the PR's tests, 33 -key cases of test_linear_orders_the_body_and_the_credential and test_notion_reads_a_request_as_real_does fail. Every new input is a row of an existing measured table.
Note: uv.lock:2 — the lock goes from revision = 5 (the base and main) to revision = 3 and rewrites the markers of httpcore2, importlib-metadata and secretstorage. uv lock --check still passes, but regenerating with the uv that wrote the base's lock would leave a diff that only adds brotli.
Prose: block
backlot/graphql/body.py:385— "Python's decoder refuses bytes after the stream, which Node leaves unread: the stream ..." repeats_unbrotli's docstring three lines above ("Node reads one stream and leaves what follows it unread"). Suggested: "# Python's decoder refuses bytes after the stream: the stream ends at the shortest prefix that finishes it, if any does."- Note, PR body, Verification — the 683-character paragraph arguing each surviving mutation could be one sentence, e.g. "The six survivors cannot change an answer:
,read as/outside IMAP, two disjoint branches swapped, a limit check the later read repeats, unjoined surrogate units, and two that never return.", or a table under the command block.
…linear-a-utf-7-body-is-read-as-iconv-lite-reads-it
…f only adds brotli
…, the docstring having said what Node leaves unread
…t read, is the decompression failure The prefix search stops at the shortest prefix that finishes the stream or that Brotli refuses, no longer at one that has given more than the limit. api.linear.app answered such a body with the decompression failure in 14 of 16 sends on 2026-10-29 (UTC), over three bodies.
…linear-a-utf-7-body-is-read-as-iconv-lite-reads-it
|
Applied in
|
khj809
left a comment
There was a problem hiding this comment.
Re-checked at 6a201eeb: every finding of the earlier review is addressed. uv.lock is back at revision = 5 and adds only brotli, and the comment in _unbrotli no longer repeats its docstring.
…s-answered-first' into linear-a-utf-7-body-is-read-as-iconv-lite-reads-it
…linear-a-utf-7-body-is-read-as-iconv-lite-reads-it
…linear-a-utf-7-body-is-read-as-iconv-lite-reads-it
Closes #849. A body on Linear's GraphQL route sent with
Content-Encoding: bris decompressed as Linear'sbody-parser2.x decompresses it, and one declaredutf-7orutf-7-imapis decoded as iconv-lite 0.7 decodes it, on Notion's routes too.What changed
Brotli (
backlot.graphql.body._unbrotli). A new runtime dependency,brotli>=1.2; the lock takes 1.2.1, the first release with wheels for Python 3.15.body_parser_textpasses abrbody through_unbrotli, whose docstring gives the answers to a stream Brotli cannot read, cut short, followed by more bytes, past the limit, and cut short past the limit with bytes Brotli cannot read after it. Notion's reading passesbrotli=Falseand still answersbrwith the encoding's 415.UTF-7 (
_utf7,_base64_units). iconv-lite 0.7'sutf-7andutf-7-imapdecoders, read offencodings/utf7.js, which is the same file in 0.7.2 and 0.7.3;_utf7's docstring sets out how a run is read. Notion's reading (backlot.notion_request) goes through the samebody_parser_text.Tests. Forty rows in
_LINEAR_READS(tests/test_linear.py), each with the bytes api.linear.app answered: fifteen withContent-Encoding: brorBR, the limit among them, and twenty-five withcharset=utf-7orcharset=utf-7-imap. Two rows in_REQUEST_ROWS(tests/test_notion.py) hold api.notion.com's answers to autf-7body.Not in this PR.
raw-bodyhands iconv-lite each chunk as it arrives, so a UTF-7 body that reaches Linear in pieces can decode otherwise where a piece ends inside a base64 run. Backlot reads the body whole, and how a request is cut into chunks on the way to Linear is not something a client controls, so there is no answer to model and no issue to file. The 413 in the row below for a cut-shortbrstream past the limit needs none either: the same body drew the decompression failure in the other sends, so nothing a client sends picks it.Why this is what the real API does
Measured. Against api.linear.app on 2026-10-27 (UTC) with an API key, each request with its own cache-buster: thirty-seven bodies, fifteen with
Content-Encoding: brand twenty-two declaredutf-7orutf-7-imap. On 2026-10-28, five more the rows hold: three that are a run holding two byte-order marks, and two runs of UTF-16 surrogates. On 2026-10-29,brstreams past the limit, each cut short and followed by bytes Brotli cannot read, as their row below counts. Some of them:Each of the forty rows sends one of these bodies, the ones at the limit padded by
padded_queryrather than with spaces, and the base branch answers thirty-four of them otherwise.Notion. Against api.notion.com on 2026-10-28 (UTC), with no cache-buster, since Notion refuses a query parameter it does not know, two bodies declared
utf-7toPOST /v1/search:Against iconv-lite. The two decoders gave the same UTF-16 units as iconv-lite 0.7.2, run in Node through
getDecoder,writeandendasraw-bodycalls them, for each of 64,644 generated byte strings in both charsets; 0.7.3 gave the same for the 6,001 of them that put a byte-order mark inside a run.Verification
pytestonuv sync --all-extras --lockedplusuv pip install --no-deps "llama-index-readers-hubspot<0.6", as CI installs: 8981 passed, 1 skipped.tests/test_mcp.py, which needs Docker's Linux containers and found no daemon here.ruff check . && ruff format --check .: clean.At
6a201eeb, onlinear-a-refused-root-argument-is-answered-firstatf39a1314.Each line this branch adds to
backlot/graphql/body.pywas mutated against each test file this branch or its base changes: operators generated from its lines, and by hand the markswriteandendreturn, a run the body ends in, a lone base64 character, a high byte outside a run, the bytes after a Brotli stream and the prefix the search stops at past the limit. The two survivors cannot change an answer:,read as/outside IMAP, where it ends the run, and two disjoint branches swapped. Two more never return.utf-7,utf-7-imapandbrrows this pull request adds totest_linear_orders_the_body_and_the_credential, and itsutf-7run intest_notion_reads_a_request_as_real_does.