Skip to content

Speed up parsing and serializing small frames - #208

Open
DicksonWu654 wants to merge 3 commits into
python-hyper:mainfrom
DicksonWu654:perf/small-frames
Open

DicksonWu654 wants to merge 3 commits into
python-hyper:mainfrom
DicksonWu654:perf/small-frames

Conversation

@DicksonWu654

Copy link
Copy Markdown

Most WebSocket traffic is small messages (chat, JSON events, pings). For those, the frame parser's fixed per-frame overhead is most of the CPU time. This adds fast paths for short frames in frame_protocol.py. Everything outside them (partial input, extended lengths, extensions such as permessage-deflate, reserved bits, invalid input) goes through the existing code before any input is consumed. Three commits:

  1. Reduce frame header and output buffer allocations. FrameDecoder.parse_header() looks the FIN/RSV/opcode fields up in a 256-entry table instead of building a new RsvBits for every frame. _serialize_frame() appends mask and payload to the fresh header bytearray for frames with a 7-bit length, instead of bytearray(header + masking_key + payload). Frames with extended lengths keep the original expression: appending is slower for large payloads on CPython 3.12.
  2. Decode complete short data frames directly. A complete, unfragmented text or binary frame with a payload of up to 125 bytes used to go through the staged parser: header parsing, a Frame, a second Frame from MessageDecoder, and a new incremental UTF-8 decoder for text, even though all the bytes are already buffered. When no extensions are in use and no frame or message is in progress, _parse_more_gen() now unmasks and decodes such a frame from the buffer and yields the same Frame. Invalid UTF-8 and wrong masking direction return to the staged parser, so error messages and close codes do not change.
  3. Parse short frame headers without temporary slices. Control and continuation frames still go through parse_header(). For a complete header with a 7-bit length, valid opcode, no reserved bits and no extensions in use, it now decodes the header in place. _parse_more_gen() also skips the staged parser when the buffer is empty. Before yielding in that case it drops its reference to the previous frame, as the staged loop does, so a delivered payload is not kept alive while waiting for data (there is a weakref test for this).

The fast paths are only taken for the stock FrameDecoder, MessageDecoder and Buffer with a bool client flag, and they are re-checked between frames, so extensions added mid-stream and replaced decoders are still honoured (there are tests for both).

Benchmarks

Each workload connects a client and a server Connection: the client sends, the server echoes every event back and the client receives it. small_text is a 48-character message, small_binary 100 bytes, coalesced_32 is 32 small messages delivered in one receive_data() call, small_text_bytewise delivers the small message one byte at a time, and deflate_small_text_echo negotiates permessage-deflate. project_bench_connection mirrors bench/connection.py (eight messages of ~125 KiB on average, fixed seed).

master and this branch were imported side by side in the same interpreter and timed in alternating ~20 ms CPU-time blocks (40 rounds in each of 6 processes, 4 on PyPy, pinned to one core). The table shows the median per-round speedup with a 95% bootstrap interval. The machine was shared with other work, so the absolute times are only indicative.

Workload CPython 3.12.3 master → branch (µs) CPython 3.12.3 speedup CPython 3.14.8 speedup PyPy 3.11.13 speedup
small_text_echo 52.5 → 32.4 1.62x [1.59, 1.64] 1.64x [1.61, 1.69] 1.15x [1.13, 1.18]
small_binary_echo 47.7 → 32.2 1.46x [1.44, 1.49] 1.45x [1.41, 1.48] 1.08x [1.06, 1.10]
coalesced_32_text_echo 1,232.7 → 749.0 1.65x [1.63, 1.69] 1.62x [1.60, 1.66] 1.07x [1.06, 1.10]
ping_pong 34.9 → 29.5 1.21x [1.18, 1.23] 1.14x [1.12, 1.17] 1.10x [1.07, 1.11]
medium_text_1k_echo 64.2 → 63.9 1.00x [0.98, 1.02] 0.98x [0.96, 1.01] 1.01x [0.99, 1.04]
large_binary_64k_echo 854.2 → 856.7 1.00x [0.99, 1.01] 0.99x [0.98, 1.01] 0.98x [0.93, 1.02]
small_text_bytewise 1,465.9 → 1,372.8 1.07x [1.05, 1.08] 1.02x [1.00, 1.04] 1.19x [1.15, 1.22]
deflate_small_text_echo 80.0 → 81.5 0.98x [0.96, 1.00] 1.01x [0.98, 1.05] 1.00x [0.98, 1.03]
project_bench_connection 5,107.6 → 5,160.0 1.00x [0.99, 1.01] 1.00x [0.98, 1.02] 1.00x [0.87, 1.23]

Frames above 125 bytes and connections with extensions take the existing path, so those workloads are unchanged within noise.

Behaviour

  • New tests cover all 256 first header bytes, output buffer ownership and length boundaries (0, 125, 126, 65535, 65536), coalesced data/control/close frames, invalid UTF-8 in a complete frame, mask direction with boolean and non-boolean client values, extensions added between frames, control frames delivered byte by byte, a replaced frame decoder, a Buffer subclass and payload release.
  • python -bb -m pytest passes on CPython 3.10, 3.11, 3.12, 3.13, 3.14 and PyPy 3.11, and each of the three commits passes on its own. mypy src/ (the lint tox env) passes. Coverage of frame_protocol.py is unchanged at 99% (the only missed line is the existing TYPE_CHECKING import).
  • I also ran master and this branch on the same randomized frame streams and compared the results: 2,000 scenarios (231,329 recorded results) on CPython 3.12 and 500 scenarios (56,310 results) on each of CPython 3.10, 3.11, 3.13, 3.14 and PyPy 3.11. Streams mix text (including multi-byte UTF-8 split across fragments) and binary messages, fragmentation with interleaved pings/pongs, close frames with valid and invalid codes and reasons, non-minimal length encodings and 64 KiB payloads, and in about 30% of cases a corruption (reserved bits, bad opcode, wrong mask bit, cleared FIN, stray continuation, oversized length, flipped bit, truncation). Each stream is fed whole, byte by byte, split at every position (or around every frame boundary for long streams) and in random chunks, through both Connection and FrameProtocol, recording every event, payload type, ParseFailed code and message and the final state. The sending side (all event types, bytes/bytearray/str payloads, client masking with a seeded key source, optional permessage-deflate) was compared the same way. All outputs were identical; as a sanity check, a deliberately broken decoder (latin-1 instead of UTF-8) was flagged on 4,645 results.
  • ruff check src (not run in CI) reports four new complexity findings (C901/PLR0912/PLR0915) for parse_header() and _serialize_frame(), on top of the 22 findings already on main. I kept the fast paths inline, since moving them into helpers costs a call per frame; happy to restructure if you prefer.

🤖 Generated with Claude Code

DicksonWu654 and others added 3 commits November 6, 2026 02:58
FrameDecoder.parse_header() rebuilt the same FIN/RSV/opcode fields,
including a new RsvBits tuple, for every frame. There are only 256
possible first bytes, so look the fields up in a precomputed table.

_serialize_frame() built the output for short frames with chained
concatenation (header + mask + payload, then another bytearray copy).
Append to the fresh header bytearray instead. Frames with extended
payload lengths and payloads of other types keep the original
expression, which is faster for large bytearrays on CPython.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A complete, unfragmented text or binary frame with a short payload went
through the staged parser: header parsing, a Frame, a second Frame from
the message decoder, and for text a fresh incremental UTF-8 decoder,
even though every byte is already buffered.

When no extensions are in use and no frame or message is in progress,
decode such a frame straight from the buffer and return the same Frame.
Everything else (partial input, fragmented messages, control frames,
extended lengths, reserved bits, wrong masking, invalid UTF-8) is left
to the staged parser before any input is consumed, so errors and close
codes are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Control frames and continuation frames still go through
parse_header(), which consumes the header in several small slices.
For a complete header with a 7-bit length and no extensions in use,
decode it in place using a 16-entry opcode table. Partial headers,
extended lengths, reserved bits, invalid opcodes and fragmented control
frames take the original path before anything is consumed.

received_frames() also skips the staged parser when the buffer is
empty. It drops its reference to the previous frame before yielding,
as the staged loop does, so a delivered payload is not kept alive while
waiting for more data.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant