Commit 4099e68
committed
Port N-gram Tilt from PR openai#1420 to SP8192+TTT stack
- Add ngram_tilt_enabled and tilt hyperparameters to Hyperparameters
- Add build_ngram_extension(): cmake-based C++ build for fused_expert_ext
- Add precompute_ngram_hints(): rank-0 computes, broadcasts to all ranks
- Integrate Tilt into eval_val_sliding_ttt scoring loop:
* Tilt applied AFTER TTT scoring (same sliding window)
* TTT gradient uses ORIGINAL NLL (not tilted)
* Tilted NLL accumulated for final score
- Track both base and tilted BPP for delta reporting
- Copy fused_expert_blend.cpp to repo root for C++ build1 parent 630d3f8 commit 4099e68
2 files changed
+528
-8
lines changed
0 commit comments