Commit 514119d
committed
R20v26: Muon 0.97 + EMA 0.997 + warmdown 0.667 (openai#1523 hyperparams)
Porting all openai#1523 hyperparams that differ from openai#1493:
- EMA_DECAY: 0.9965 -> 0.997 (stronger smoothing)
- WARMDOWN_FRAC: 0.72 -> 0.667 (shorter warmdown)
- Muon 0.97 (kept from previous best)1 parent 47acbd9 commit 514119d
1 file changed
+1
-1
lines changed
0 commit comments