Repository navigation
Description
Context
DESIGN.md section 10.2: "Characters and content items are columnar buffers that NumPy and Arrow read without copying; records are views over them." #606 gave the paths of a page columns, Page.paths(), in the layout agreed there, and built them in the binding from the interpretation that Page.content() memoizes (section 5), as its Paths asked. That interpretation keeps one tuple per path in the engine's content model, ContentItem::Path: 288 bytes of enum and three heap allocations for the verbs, the coordinates, and the stroke's paint. The page memo holds those tuples after the call, and building them dominates what reading a CAD page costs once the Python objects are gone.
On the 200,000-line page of #606, on Windows 11 x86-64 (evidence/path-columns/results/windows-amd64.md):
Page.paths()takes 0.25 s, a little longer than rendering the page (0.21 s), against 4.0 s forContent.paths. Once the interpretation is memoized, building the columns takes 36 ms of that.- The columns hold 224 bytes a path: 118 for the path and 53 for each of its two segments.
- The call grows the peak resident set by 581 bytes a path and leaves 693 held. About 470 of those 693 are the memoized tuples.
Goal
The content collector records paths as columns in the engine, the page memo holds them, and Page.paths() hands the memo's buffers to Python without a copy, so that reading the paths of a CAD page costs tens of bytes per segment in all, not only in the columns that Python holds.
Scope
In:
crates/pyropdf-engine/src/extract/model.rs: a columnar path table that the collector appends to, with the fill-and-stroke merge and theclipfilter of today's records, and aPageContentwhose items refer to its rows.crates/pyropdf-python/src/buffers.rsanditems.rs:PathColumnsover the memoized table, andPage.content()'s path tuples built from it.
Out:
- The public layout of
Paths, agreed in Read the paths of a page as columnar buffers #606. - Text runs and images.
- Making the
Content.pathsrecords views over the columns.
Paths
crates/pyropdf-engine/src/extract/model.rs, crates/pyropdf-engine/src/extract/mod.rs, crates/pyropdf-engine/src/extract/tests.rs, crates/pyropdf-python/src/buffers.rs, crates/pyropdf-python/src/items.rs, and evidence/path-columns/.
Acceptance
-
tests/test_path_columns.pyandtests/test_content_model.pypass unchanged, andmax_elementsrefuses withcontent_element_countat the same bounds -
evidence/path-columns/measure.pyrerun showscolumnscosting at most a few hundred bytes of peak and held memory per line on the 200,000-line page, with the results committed
Notes
- The collector merges a stroke into the fill before it when both paint the same segments (
record_path), andfinishdrops the items outsideclipafter the budget has counted them; the table needs both.
Activity
yhay81 commentedon Nov 4, 2026
Claiming: branch
claude/615-engine-path-columns.yhay81 commentedon Nov 4, 2026
Handoff: PR #642 on
claude/615-engine-path-columns.PathTable, the page memo holds it, andPage.paths()shares its buffers without a copy. Items refer to rows.columnson the 200,000-line page went from 583 to 156 B of peak and from 689 to 310 B held per path.paths()from the memo went from 51 ms to 0.03 ms.tools/check.py --lockedpasses on the tree merged withmainat 093c9b3, andfuzz/builds. CI on the PR is pending and will include Keep a CFF face's hints when a save compacts it #611.in-progressand report so that Measure the exact boxes of non-embedded CJK fonts on their fallback faces #624 can start.Record the paths of the content model as columns in the engine (#615) (…