Skip to content

Commit 7fc9b7c

Browse files
committed
Upgrade synthesis: add SLOT-24 (quad-stack) — Pre-Quant TTT + Val-Calib GPTQ + SLOT-24
Replaces the triple-stack (Pre-Quant TTT + Val-Calib GPTQ + Eval-Time Legal TTT) with a quad-stack that supersedes the legal TTT path with SLOT-24, ported from PR openai#1488 / PR openai#1313. Four val-data adaptations stacked for the first time: 1. Pre-Quant AdamW TTT — 11 epochs, freeze_blocks=0 (Track A) 2. Val-Calibrated GPTQ — Hessian H=X^T X from val activations (Track A) 3. SLOT-24 — per-window hidden delta + logit bias on the frozen post-quant model, 24 cosine-decayed AdamW steps, throwaway parameters 4. (Optional) Eval-Time Legal Score-First TTT — disabled by default; SLOT supersedes it within the eval budget. Set SLOT_ENABLED=0 TTT_ENABLED=1 to fall back. Code changes vs the previous synthesis commit: - GPT class: split forward_logits into forward_hidden + compute_logits so SLOT can add the per-window delta to the hidden state without re-running the transformer stack. - New eval_val_slot function ported from PR openai#1488 (per-window AdamW with cosine LR decay, stride masking, score-after-delta). - run_evals: wires SLOT on a fresh post-quant model copy, gated by SLOT_ENABLED. Disables legal TTT by default. - New hyperparameters: SLOT_ENABLED, SLOT_STEPS, SLOT_LR, SLOT_LR_MIN, SLOT_BATCH_SEQS, SLOT_EVAL_STRIDE. Folder renamed: 2026-04-09_PreQuantTTT11_ValCalibGPTQ_LegalEvalTTT_Synthesis -> 2026-04-09_PreQuantTTT11_ValCalibGPTQ_SLOT24_Quad_Synthesis Time budget: ~530s of 600s eval used (590s train + 190s prequant TTT + 10s val-calib GPTQ + 80s sliding eval baseline + 250s SLOT-24). Code: 2322 lines (vs 2039 in PR openai#1487 base, +283 added). py_compile clean. README rewritten as user's submission with compact credits section.
1 parent d50b05c commit 7fc9b7c

8 files changed

Lines changed: 476 additions & 335 deletions

File tree

records/track_10min_16mb/2026-04-09_PreQuantTTT11_ValCalibGPTQ_LegalEvalTTT_Synthesis/README.md

Lines changed: 0 additions & 149 deletions
This file was deleted.

records/track_10min_16mb/2026-04-09_PreQuantTTT11_ValCalibGPTQ_LegalEvalTTT_Synthesis/VALIDATION.md

Lines changed: 0 additions & 130 deletions
This file was deleted.

records/track_10min_16mb/2026-04-09_PreQuantTTT11_ValCalibGPTQ_LegalEvalTTT_Synthesis/submission.json

Lines changed: 0 additions & 39 deletions
This file was deleted.

0 commit comments

Comments
 (0)