fix(kernel): make Triton top-k/top-p sampling exact - #329
Conversation
|
Thanks for folding #345 in and for the follow-up work — the per-device I'm planning to merge this PR instead of #345. One change before that: the tie clipping in |
|
LGTM. |
Traz o commit oficial novo: - fix(kernel): make Triton top-k/top-p sampling exact (FlashML-org#329) Sem conflito com o trabalho local do fork -- confirmado via 'git merge-tree --write-tree', arquivos tocados nao se sobrepoem.
…lashML-org#336 hf shard download Cherry-picked from FlashML-org/FreeToken 092ce4a; FlashML-org#343 is redundant with the fork's compressed-tensors NVFP4 detection and FlashML-org#319 conflicts with the kernel backend, both left out. Box tests 408 passed with the known order-dependent parity failure (FlashML-org#18). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A88MCbnLtwsFSHmqwuJezY
Brings in exact Triton top-k/top-p sampling (FlashML-org#329), nvfp4 detection behind mixed-precision compressed-tensors (FlashML-org#343), GLM-5.3-Flash config test, issue templates and README/CONTRIBUTING edits. Trial merge showed no conflicts; README.md and models/config.py were the only overlapping files. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FKGJYhK5m18zMnjm5nD8TE
Summary
Dropping the token-id tie clipping also removes its per-CTA tie workspace and one synchronization stage from the fused path. This keeps the final PR to one production file; no test files are included.
Validation
Credit
This revision builds directly on #345. Thank you, @jason-fxz, for taking the original correctness report seriously, replacing the fragile estimator with a much stronger exact design, and carefully reviewing #329. The core single-launch histogram approach here is your work; the follow-up changes retain its exactness while making device handling and fallback behavior more robust.