unicode: Unicode 17.0.0 is the width and grapheme authority - #32
Merged
Merged
Conversation
…d UCD data pinned by SHA-256, generated tables, the library, and its generator vendor/unicode/17.0.0 holds the Unicode Character Database and emoji data files the tables derive from. Their SHA-256 digests are recorded in tools/unicode/generate.zig and in the generated manifest.zig; PIN.md points at those. tools/unicode/fetch.sh is the only path that touches the network; a normal build and `zig build unicode-check` (regenerate and byte-compare the tables) run offline. Every symbol main's callers use is still exported with the same signature. Two deviations from the feature branch this is extracted from. wrapLine and legacyValidPrefix are restored verbatim from 6d65f9b^ because main's src/core/markdown/render/table.zig still calls wrapLine. The three nextGlyph callers (src/tui/app.zig, src/tui/selection.zig, src/core/markdown/render/frontmatter.zig) gained a one-byte floor on their advance, because the new nextGlyph reports a malformed byte as an empty slice and an unfloored loop would never move past it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012r9nF92bG3LRf1DC3r933M
This was referenced Sep 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This lands Unicode 17.0.0 as the width and grapheme authority: the vendored UCD and emoji data under
vendor/unicode/17.0.0(SHA-256 digests recorded intools/unicode/generate.zig), the generated tables undersrc/lib/unicode/generated, the librarysrc/lib/unicode.zig, its generator, and azig build unicode-checkstep that regenerates the tables offline and byte-compares them against the committed ones.tools/unicode/fetch.shis the only path that touches the network.It is the first slice of #31, split out so the data-heavy part can be reviewed and merged on its own. #31 is now based on this branch and retargets to
mainautomatically once this merges. Every symbol main's callers use keeps its signature.Deviations from #31
wrapLineandlegacyValidPrefixare restored verbatim, because main's markdown table renderer still callswrapLine.nextGlyphcallers (src/tui/app.zig,src/tui/selection.zig,src/core/markdown/render/frontmatter.zig) gained a one-byte floor on their advance. The newnextGlyphreports a malformed byte as an empty slice, which would otherwise hang those loops on invalid UTF-8. Each site has a regression test.Behavior notes
codepointWidthnow gives emoji-presentation code points width 2 and C1 controls width 0 (callers insrc/exportand the legacy mermaid canvas).Verification
zig build,zig build test(1663 passed, 3 skipped),zig build unicode-check,zig fmt --check, all green on this branch alone.🤖 Generated with Claude Code
https://claude.ai/code/session_012r9nF92bG3LRf1DC3r933M