Skip to content

Conversation

@simonklee
Copy link
Contributor

The is ascii only checks isn't exactly what the name implies. It strictly enforces printable ASCII (32-126), explicitly excluding control characters like tabs (\t) and newlines.

This provides a stronger guarantee than typical 7-bit ASCII checks: if isAsciiOnly is true, every byte is exactly 1 column wide.

However, we ignored this and still running O(N)
width loops even on the fast path. This patch deletes those loops entirely:

  1. If it's guaranteed printable ASCII, the display width is identical to text.len. We don't need to iterate N bytes just to add 1 N times.

  2. Since width maps 1:1 to byte index, the wrap position is simply min(text.len, max_width). We don't need to scan the string to find where it overflows.

The obvious risk here is tabs (byte 9), which are strictly ASCII but variable width.

But since isAsciiOnly checks val >= 32, so it returns false for \t. This forces tabbed content into the slow Unicode path where tab_width is handled properly.

I also considered if it was possible for FFI consumers to pass isAsciiOnly=true for strings with tabs or newlines. But since the public API doesn't expose isAsciiOnly directly, and instead derives it via utf8.isAsciiOnly(), which returns false for empty strings and control characters, this optimization is safe and transparent to external users.

The is ascii only checks isn't exactly what the name implies. It
strictly enforces printable ASCII (32-126), explicitly excluding
control characters like tabs (`\t`) and newlines.

This provides a stronger guarantee than typical 7-bit ASCII checks:
if `isAsciiOnly` is true, every byte is exactly 1 column wide.

However, we ignored this and still running O(N)
width loops even on the fast path. This patch deletes
those loops entirely:

1. If it's guaranteed printable ASCII, the display width is
   identical to `text.len`. We don't need to iterate N bytes just to
   add 1 N times.

2. Since width maps 1:1 to byte index, the wrap position is simply
   `min(text.len, max_width)`. We don't need to scan the string to
   find where it overflows.

The obvious risk here is tabs (byte 9), which are strictly ASCII but
variable width.

But since `isAsciiOnly` checks `val >= 32`, so it returns false for
`\t`. This forces tabbed content into the slow Unicode path where
`tab_width` is handled properly.

I also considered if it was possible for FFI consumers to pass
`isAsciiOnly=true` for strings with tabs or newlines. But since
the public API doesn't expose `isAsciiOnly` directly, and instead
derives it via `utf8.isAsciiOnly()`, which returns false for
empty strings and control characters, this optimization is safe
and transparent to external users.
@simonklee simonklee force-pushed the perf-utf8-ascii-invariant branch from 9c432af to 6d32b43 Compare January 9, 2026 21:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant