Commit 7fc9b7c
committed
Upgrade synthesis: add SLOT-24 (quad-stack) — Pre-Quant TTT + Val-Calib GPTQ + SLOT-24
Replaces the triple-stack (Pre-Quant TTT + Val-Calib GPTQ + Eval-Time Legal TTT)
with a quad-stack that supersedes the legal TTT path with SLOT-24, ported from
PR openai#1488 / PR openai#1313.
Four val-data adaptations stacked for the first time:
1. Pre-Quant AdamW TTT — 11 epochs, freeze_blocks=0 (Track A)
2. Val-Calibrated GPTQ — Hessian H=X^T X from val activations (Track A)
3. SLOT-24 — per-window hidden delta + logit bias on the frozen post-quant
model, 24 cosine-decayed AdamW steps, throwaway parameters
4. (Optional) Eval-Time Legal Score-First TTT — disabled by default; SLOT
supersedes it within the eval budget. Set SLOT_ENABLED=0 TTT_ENABLED=1
to fall back.
Code changes vs the previous synthesis commit:
- GPT class: split forward_logits into forward_hidden + compute_logits so
SLOT can add the per-window delta to the hidden state without re-running
the transformer stack.
- New eval_val_slot function ported from PR openai#1488 (per-window AdamW with
cosine LR decay, stride masking, score-after-delta).
- run_evals: wires SLOT on a fresh post-quant model copy, gated by
SLOT_ENABLED. Disables legal TTT by default.
- New hyperparameters: SLOT_ENABLED, SLOT_STEPS, SLOT_LR, SLOT_LR_MIN,
SLOT_BATCH_SEQS, SLOT_EVAL_STRIDE.
Folder renamed: 2026-04-09_PreQuantTTT11_ValCalibGPTQ_LegalEvalTTT_Synthesis
-> 2026-04-09_PreQuantTTT11_ValCalibGPTQ_SLOT24_Quad_Synthesis
Time budget: ~530s of 600s eval used (590s train + 190s prequant TTT + 10s
val-calib GPTQ + 80s sliding eval baseline + 250s SLOT-24).
Code: 2322 lines (vs 2039 in PR openai#1487 base, +283 added). py_compile clean.
README rewritten as user's submission with compact credits section.1 parent d50b05c commit 7fc9b7c
8 files changed
Lines changed: 476 additions & 335 deletions
File tree
- records/track_10min_16mb
- 2026-04-09_PreQuantTTT11_ValCalibGPTQ_LegalEvalTTT_Synthesis
- 2026-04-09_PreQuantTTT11_ValCalibGPTQ_SLOT24_Quad_Synthesis
Lines changed: 0 additions & 149 deletions
This file was deleted.
Lines changed: 0 additions & 130 deletions
This file was deleted.
Lines changed: 0 additions & 39 deletions
This file was deleted.
0 commit comments