Skip to content

Add a fast path that doesn't include normalized chunks in tokenize#11017

Open
jsignell wants to merge 1 commit intopydata:mainfrom
jsignell:just-use-token
Open

Add a fast path that doesn't include normalized chunks in tokenize#11017
jsignell wants to merge 1 commit intopydata:mainfrom
jsignell:just-use-token

Commits

Commits on Dec 15, 2025