[yhay81]

Closes #600

Text in a CJK font that a PDF does not embed rendered blank: hayro asks its font resolver for such a font, and the default resolver answers with a Standard 14 font, which has no CJK glyphs. This adds a resolver for those queries, ported from pylopdf 4ff4a266 (v0.16.0): pick_cjk_fallback, the name hints, and interpreter_settings of rust/src/document/, _FONT_PROVIDER_SPECS and _bundled_font_providers of src/pylopdf/_fonts.py, and the FallbackFont API of src/pylopdf/_fallback_fonts.py, with the rendering cases of tests/test_locale_fonts.py. These files are the same at the freeze commit ebb01d3c.

What changed

• Engine (crates/pyropdf-engine/src/render/fallback.rs): Fallback holds a sans and a serif face for each of ja, ko, zh-CN, and zh-TW. Its resolver takes the language from the character collection (Adobe-Japan1, Korea1, GB1, CNS1), or for Adobe-Identity from the BaseFont name, and the serif face for a serif name such as a Mincho; every other query goes to hayro's default resolver. A face is a loaded Font or an installed font file, which is read once, the first time that a page needs it. raster, batch, and svg take the faces, and the thread's render cache is keyed by them as well as by the bridge (DESIGN.md §5).
• Binding: each _Document holds its faces (set_fallback_faces), and Page.pixmap, Document.render, Page.svg, and Annotation.pixmap render with them. Unlocking keeps them.
• Python: pyropdf.FallbackFont(sans=..., serif=...) and doc.fallback_fonts, a mapping by language that holds the faces of the installed pylopdf-fonts-* packages until it is edited; {} renders without fallback fonts, and del returns to the installed extras. They are not saved, and text extraction does not use them (out of Render text in non-embedded CJK fonts with fallback faces #600's scope).
• Packaging: the extras jp, ko, zh-cn, and zh-tw install pylopdf-fonts-*>=0.1.1,<0.2; the dev group installs them for the tests.
• Docs: the rendering guide and Getting started, in English and Japanese, name the extras; the API reference is regenerated.

Acceptance

• With pylopdf-fonts-jp installed, the page above draws its three glyphs in pixmap() and svg(), and a Mincho BaseFont picks the serif face: test_the_installed_japanese_faces_draw_the_page (both descendant types and both CMaps, with no empty glyph outline; the page draws 1,441 dark pixels through hayro 0.9's glyph cache, and without it the 1,437 of the issue's pylopdf 0.16.0 with jp) and test_a_mincho_font_takes_the_serif_face.
• GB1, CNS1, and Korea1 pages pick their language's face when its package is installed: test_each_collection_takes_the_faces_of_its_language, which also checks that the other languages' faces never draw them.
• Without an installed face, rendering is as now, and the documentation names the extras: test_without_installed_faces_the_text_renders_blank; docs/en/rendering.md#fonts-that-a-document-does-not-embed, its Japanese page, and Getting started.
• The differential tests against frozen pylopdf (DESIGN.md section 12) include such a page: tests/assets/text/nonembedded-cjk.pdf, the three pages above, MS-Mincho, and a GB1, CNS1, and Korea1 page, in evidence/phase1-parity and tests/test_parity.py. Without hayro 0.9's glyph cache (Decision: hayro 0.9's glyph cache changes 152 of 163 corpus renders; keep it or turn it off for 0.2.2 #622), all 7 render byte for byte as frozen pylopdf with its CJK extras renders them; through it, 4 are close under the harness's tolerance and 3 differ in 0.6 to 0.8% of their pixels, as the corpus's other text pages do.

Evidence

evidence/phase1-parity ran again on 2026-11-03 at d2715c8, this branch merged with main on hayro 0.9 (Windows, CPython 3.14.8), with frozen pylopdf ebb01d3c on hayro 0.8, built with [jp,ko,zh-cn,zh-tw]; its font files have the same SHA-256 as those of the PyPI packages that pyropdf's dev group installs.

Renders changed with #609, apart from this PR. Of the 163 rasters, 11 match byte for byte, 7 are close under the tolerance, and 145 are not. The cause is RenderSettings::glyph_caching, on by default in hayro 0.9, which #609 leaves on when it turns hinting off: with glyph_caching: false, measured and not committed, 162 match byte for byte and the last differs in 495 pixels by at most 5. Before the merge, on hayro 0.8, all 163 matched (e5b0d10). The owner chose on #18 that 0.2.2 changes no render, so #622 asks whether to keep the cache, which makes rendering about 1.8 times as fast. This PR does not change it, and records it as the cause hayro-glyph-cache; whichever of this PR and #622's change lands second measures again.

The run also records what changed since the run of 2026-10-28:

• fallback-exact-boxes: words(exact_bboxes=True) differs on pages 2 to 7 of the fixture, by up to 7.28 points, because pylopdf's extraction measures the outlines of its fallback faces and pyropdf's extraction does not use them. The owner chose to leave this out of 0.2.2 and of this PR; Measure the exact boxes of non-embedded CJK fonts on their fallback faces #624 follows it.
• heading-above-columns: page 2 of patent-us223898.pdf keeps its headings whole since Headings above two columns are split into the columns in page.text() #460, which landed after the previous run.
• The highlight of pdfium-links-highlights-annots.pdf renders alike since Draw annotations stored without an appearance at render time (#169) #172, so its cause is retired; without the glyph cache, the page matches byte for byte.
• pdfium-ocg-invisible.pdf, which was added after the previous run, is measured now, and matches on every output.
• compare.py read a span's text from text only; the spans of layout(chars=True) hold characters, which it now joins. No difference had reached that path before.

PYROPDF_PARITY_PYLOPDF=target/parity/pylopdf-env/Scripts/python.exe uv run pytest tests/test_parity.py passes, with the redump of the fallback pages.

Notes for review

• DESIGN.md §10.1 says that output depends on "the document, the call's arguments, and the fonts passed in". The installed extras now set the default faces, as pyropdf[ocr] sets OcrEngine()'s models. The owner approved the wording on Render text in non-embedded CJK fonts with fallback faces #600; it is PR DESIGN.md: output depends on the installed CJK font extras (#625) #626 (DESIGN.md: output depends on the installed CJK font extras #625), which merges with or after this PR, and this PR does not change DESIGN.md.
• Merged with main (hayro 0.9, Move to hayro 0.9 with LaurenzV/hayro#1734 patched in, and read streams that name a file #609): the conflicts were the InterpreterSettings literals of render/mod.rs and render/svg.rs, which take render_content from Move to hayro 0.9 with LaurenzV/hayro#1734 patched in, and read streams that name a file #609 and ..fallback.settings() from this PR, and the dev group's entry in uv.lock, which uv lock leaves as resolved. The resolver's types have the same paths in hayro 0.9 (hayro_interpret::font::{FallbackFontQuery, FontData, FontQuery} and hayro_cmap::CidFamily). hayro 0.9 also caches resolved programs by query in its RenderCache, which the thread's cache, keyed by the faces, already keeps apart.
• The OCR layer's font is named PyroOCR-Gothic, which the name hints read as Japanese, so with jp installed, rendering a page with an OCR layer reads the Japanese sans face once per process; the layer stays invisible.
• Four test docstrings said that pyropdf has no doc.fallback_fonts or no font extras (test_search.py, test_ocr_layer.py, test_acroform.py, test_canvas_text.py); they now say what is true. tests/corpus.py's comment on ADDED no longer names the parity evidence, which measures that file now.

Checks

python tools/check.py --locked on Windows 11 with CPython 3.14.8, at 0bab51a, this branch merged with main at 614fb83: sync, pytest (6,613 passed, 691 skipped), ruff, ruff format, ty, cargo fmt, clippy, and cargo test all passed. tests/test_parity.py passes with PYROPDF_PARITY_PYLOPDF set, which redumps the fallback pages against frozen pylopdf. The audit tools were not run: this PR changes neither Cargo.lock nor the workflows.

CI's Fuzz bridge job failed at fbbc204: the fuzz targets, which are not members of the workspace that tools/check.py checks, called render::raster without its new fallback argument. c0ac774 passes no faces in the bridge and interpret targets, and cargo check --manifest-path fuzz/Cargo.toml --bins builds them all after the merge. The evidence/ crates stay as they were measured: since #609, whose patch table only the root and fuzz/ manifests repeat, evidence/edit-render/split does not build whatever it passes.

Ready for the owner's review. It has no merge authorization.

🤖 Generated with Claude Code