markdown: the markdown, export, and TUI renderers measure and clip text through the Unicode 17 library - #34
Merged
Merged
Conversation
…xt through the Unicode 17 library — one grapheme iterator replaces the per-file width loops The markdown render model moves into render/line.zig (SpanStyle, Span, Line, Rendered) with render/types.zig re-exporting it. render/geometry.zig is the strict width wrapper the table, front matter, inline, code, and rule paths use; line.zig, wrap.zig, and builder.zig call the library directly. Every measure returns error.InvalidUtf8 or error.DisallowedControl instead of guessing a byte width, and the builder validates each finished line as a whole, so malformed input is rejected once at render time rather than tolerated by each wrap, table, and front matter loop. Code blocks and rules leave blocks.zig for render/code.zig and render/rules.zig. Export layout, plain, and PNG walk unicode.Iterator graphemes; the glyph sheet measures with rawDisplayWidth. TUI selection, copy preview, toast, and metadata use PreparedLine and the same iterator. The legacy Mermaid canvas draws text by authority width and refuses non-scalar or tab text, and its renderers get their own test root (src/core/mermaid/legacy_test.zig, `zig build test-mermaid-legacy`). src/export/font.zig adds six required glyphs (U+25B3, U+25B7, U+25BD, U+25C1, U+25CB, U+2715). src/core/theme.zig and src/core/theme/ (comment stripping and literal reflow only), src/export/png_encode.zig, src/export/surface.zig, and the legacy mermaid parser.zig, types.zig (besides its `text` module import), and state/render.zig change comments only. build.zig adds the `text` and `unicode` modules to the executable and unit test roots and wires the legacy Mermaid test root into `zig build test`. User-visible behavior changes in this slice: - The legacy Mermaid canvas (sequence, class, ER, state) draws a label only when every grapheme is a single scalar; a label containing a multi-scalar grapheme (emoji with VS16, flags, ZWJ sequences, combining marks) or a tab is dropped silently, where the previous canvas drew it. - Markdown documents containing a Grapheme_Cluster_Break=Control code point (soft hyphen U+00AD, zero-width space U+200B, LRM/RLM, word joiner) are rejected whole with `error: DisallowedControl`, in every format and in the TUI; previously they rendered. - Malformed UTF-8 anywhere in a document is rejected with `error: InvalidUtf8`; previously it was tolerated byte-wise. - Table cells are re-tokenized on spaces and re-joined with single spaces, so runs of spaces inside a cell (including inside inline code) collapse. - Link and image widths are measured from rendered text, so multibyte URLs count columns, not bytes; tables with such links come out narrower. - Tabs in front matter expand to four-column stops in every style, including raw on the terminal. - Options.for_export is now inert for markdown; sanitizeFrontMatter, sanitizeText, decor.slotPtr, inline.inlinesDisplayWidth, and wrap.isWhitespace are deleted. Extracted from the rail-renderer branch as the second slice; src/core/mermaid_v2 is untouched here. src/cli/input.zig and src/cli/input_test.zig are pulled in whole because they must import lib/text.zig by module name once the `text` module exists, or the file belongs to two modules and the build fails. The one-byte floors at the nextGlyph callers in tui/app.zig, tui/selection.zig, and markdown/render/frontmatter.zig are gone with those loops; malformed UTF-8 now exits with error.InvalidUtf8 from the builder and geometry path, verified on every front matter style in well under a second. The three floor regression tests were dropped with the loops they guarded and are replaced by tests that expect error.InvalidUtf8 in selection.zig, metadata.zig, and builder.zig. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012r9nF92bG3LRf1DC3r933M
…, one prepare pass per line, one status-message helper Cleanup pass over the Unicode 17 render-model change; no output changes. - Line.joinedText replaces five hand-written span concatenation loops (selection, plain export, PNG layout, line preparation, a test). - Line.prepareOwned measures a tab-free line in place instead of building a prepared copy, a byte map, and re-duplicating every span; the blockquote paths no longer prepare a line before finish prepares it again. - Selection: one overlap scan serves both highlight and copy; rows outside the selection return before allocating; Line replaces anytype and the test doubles. - Metadata overlay prepares each entry once per frame and reuses the theme's panel style type instead of mirroring it. - Table alignment measures a cell once; the offset search remains only for cells containing a tab. Natural widths fill the widths array directly. - Rules, code frames, and the mermaid debug dump call Builder.appendRepeated and one debug-line helper instead of private repeat and format loops. - TUI: one reportMeasureError replaces five copies of the same switch. - Removed: the unused Options.for_export, a stale export error arm, the PNG exporter's duplicate nextGrapheme, three validation-only width calls, and empty branches in the paragraph and state-diagram code. - Export tests share one ChildEnv helper for the temporary config home. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BrTcgx7ZfWmNowJ1RXAwfv
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Re-opens #33 against
main(#33 merged intounicode-17-authorityafter that branch had already landed as #32). Rebased onto main; same two commits.Second slice of #31, stacked on #32. The markdown, export, and TUI renderers, the theme files, and the legacy Mermaid canvas now measure and clip text through the Unicode 17 library. One grapheme iterator replaces the per-file width loops.
src/core/mermaid_v2is untouched here; #31 is now based on this branch and retargets automatically once it merges.What moves
render/line.zig;render/geometry.zigis the strict width wrapper the table, front matter, inline, code, and rule paths use.blocks.zigforrender/code.zigandrender/rules.zig.PreparedLine.zig build test-mermaid-legacy, folded intozig build test.build.zigadds thetextandunicodemodules to the executable and test roots.src/cli/input.zigswitches to thetextmodule import for the same reason.User-visible behavior changes (all of these are how #31 behaves today; they are listed so they get reviewed on their own)
error: DisallowedControl, in every format and in the TUI. They rendered before.error: InvalidUtf8. Before, it was tolerated byte-wise. This supersedes the one-byte loop floors unicode: Unicode 17.0.0 is the width and grapheme authority #32 added; those loops no longer exist.Verification
zig build,zig build test(1680 passed, 3 skipped),zig fmt --checkgreen on this branch alone.error: InvalidUtf8immediately in every style.🤖 Generated with Claude Code
https://claude.ai/code/session_012r9nF92bG3LRf1DC3r933M