Reuse bounded CPU raster workers with document-owned cleanup #715

Context

#365 requires corpus rendering within 1.5 times MuPDF at 72 and 144 dpi. A release-wheel prototype using four vello_cpu workers and resetting the context between consecutive pages measured 1.04 and 1.05 times MuPDF in the official runner. A six-way interleaved run gave 1.038 and 1.030 for the best totals. All 2,340 sequence checks passed. On the same six-core Windows host the ordinary baseline in that interleaved run was 1.873 and 1.704. Evidence, raw samples, source patch and reproduction: report.

The prototype is unsuitable as shipped: a partial document read retained four native threads after close, concurrent readers can multiply pools, and vello's pool constructor unwraps OS thread-creation failure. Some individual documents also exceed 1.5. DESIGN.md sections 3 (F1/F4/F6/F9), 4.6, 6, 9, 11 and 12 apply.

Goal

Deliver a bounded default native raster path that amortizes worker setup where worthwhile, preserves exact pixels and existing public behavior, and releases or safely reuses resources under concurrent use. Verify the shipping artifact independently of the experimental wheel.

Scope

In: page raster context selection and lifecycle, document close/drop/in-flight calls, bounds on worker count and scratch, serial fallback, native and unsupported targets, and end-to-end validation. Keep batch concurrency within its existing limits. Choose thresholds from measurements. Out: changing antialiasing, glyph-cache/hinting defaults, benchmark workloads, publishing upstream without permission or changing the #365 target.

Paths

• Cargo.toml, Cargo.lock, fuzz/Cargo.toml and fuzz/Cargo.lock if dependency resolution changes
• crates/pyropdf-engine/Cargo.toml and crates/pyropdf-engine/src/render/
• crates/pyropdf-python/src/document.rs, pixmap.rs and annotations.rs (including raster resource ownership and teardown)
• tests/ for raster, document lifetime and concurrent behavior
• evidence/pymupdf-comparison/ for reproducible validation and measurements

Acceptance

• Use released interfaces where possible. Keep concurrency bounded across simultaneous callers and retain the serial path for small jobs, limited platforms and exhausted worker capacity. Preserve explicit Document.render(workers=...) semantics and avoid nested worker pools.
• Release a partial read's idle workers and scratch on document close/drop, including close on another thread. An in-flight render either finishes consistently or returns the established typed error. Reuse across pages, version changes, sizes and failed calls cannot leak graphics state or keep a closed document alive.
• Handle unavailable parallelism and pool initialization failure through a safe serial fallback or a typed refusal. If a dependency change is necessary, draft it locally and request permission before publication. No unapproved dependency fork.
• Test page, batch and annotation callers, resource limits and cancellation, single-page and discontinuous reads, post-close/drop worker accounting, and concurrent independent documents. Verify exact bytes for the 1,872 corpus configurations and the mixed-order and revisit cases.
• Run the project's full locked checks and relevant dependency/audit/wasm checks. Compare actual release wheels at 72/144 dpi with the official runner and balanced ordering. Report peak memory, retained workers, and aggregate plus per-document results, including regressions. Link remaining Rendering speed: close MuPDF's lead on the corpus #365 work explicitly.

Notes

The existing thread-local render cache retains a bridge. Storing a native worker context there reproduces the after-close retention recorded above. Consider a document-owned, synchronized checkout/return slot plus a process-wide worker cap, separate from the borrowed hayro cache. The prototype's environment variable is measurement infrastructure. Avoid shipping it. The pinned hayro derives zero workers for nested contexts; confirm all entry points with the final feature configuration. Baseline, prepared-outline, batching and multi-worker artifacts are independent. This task does not depend on merging draft #709.