Record: 11L Int6 QAT + SmearGate + OrthoInit + SWA + TTT (val_bpb=1.1478)#150

Draft

yahya010 wants to merge 7 commits intoopenai:mainfrom

yahya010:submission/v12-next

Commits on Mar 20, 2026

Record: Int6 QAT + BigramHash + MLP 1344 (val_bpb 1.1593)

yahya010
and
claude
committed
Record: Int6 QAT + BigramHash + Muon WD (mean val_bpb=1.1577)

yahya010
and
claude
committed
Record: 11L Int6 QAT + SmearGate + OrthoInit + SWA + TTT (val_bpb=1.1478)

yahya010
and
claude
committed
Update: FA3 + fullgraph=True (val_bpb=1.1454)

yahya010
and
claude
committed

Commits on Mar 21, 2026

Update: no-SWA + TTT freeze=0 (val_bpb=1.1414)
yahya010
committed
Update: no-QAT + SWA(29 snapshots) + TTT freeze=0 (val_bpb=1.1393)
yahya010
committed

Commits on Mar 22, 2026

Add key scripts: v21 (best 1.1393), v24 (SOTA-based), Modal runner

yahya010
and
claude
committed