diff --git a/records/track_10min_16mb/2026-04-04_MuonEqR_3LayerRecurrence_WD095_MLR022_AllInt6/README.md b/records/track_10min_16mb/2026-04-04_MuonEqR_3LayerRecurrence_WD095_MLR022_AllInt6/README.md
new file mode 100644
index 0000000000..2dd2f9e7cf
--- /dev/null
+++ b/records/track_10min_16mb/2026-04-04_MuonEqR_3LayerRecurrence_WD095_MLR022_AllInt6/README.md
@@ -0,0 +1,78 @@
+## Record: MuonEq-R + 3-Layer Recurrence + WD=0.095 + MLR=0.022 + All-Int6 (val_bpb: 1.0900)
+
+**val_bpb = 1.0900** (3-seed mean, std 0.0005) | **2.5078 nats** | **~15.96 MB** | 8xH100 SXM, 590s train + ~81s eval | No TTT
+
+Built on [PR #1218](https://github.com/openai/parameter-golf/pull/1218) by @clarkkev.
+
+Previous: [PR #1218](https://github.com/openai/parameter-golf/pull/1218) (1.0979) -> [PR #1285](https://github.com/openai/parameter-golf/pull/1285) (1.0912) -> this (1.0900)
+
+### Changes from PR #1285
+
+| | PR #1285 | This |
+|---|---|---|
+| val_bpb | 1.09124 | **1.08995** |
+| Recurrence | Layers 4,5 (2-layer) | **Layers 3,4,5 (3-layer)** |
+| Weight decay | 0.090 | **0.095** |
+| Matrix LR | 0.020 | **0.022** |
+| Everything else | Same | Same |
+
+### Key Innovations
+
+1. **3-Layer Depth Recurrence** — Layers 3, 4, and 5 are repeated (RECUR_LAYERS=3,4,5), creating 14 virtual layers from 11 physical. MLP weights fully shared. ~0.0005 BPP improvement over 2-layer recurrence.
+
+2. **WD=0.095 + MLR=0.022 Synergy** — Higher weight decay (0.095 vs 0.090) compresses weights better, while slightly higher matrix LR (0.022 vs 0.020) recovers quality. The net effect is better BPP at the same artifact budget. The 3-layer recurrence needs WD≥0.093 to fit all-int6 under 16MB.
+
+3. **MuonEq-R + All-Int6 GPTQ** — Row-normalized Muon optimizer with all 66 layers at int6 precision (carried from PR #1285).
+
+### Configuration
+
+```bash
+NCCL_NET=Socket DATA_DIR=./data SEED=42 \
+MIXED_QUANT=1 N_INT6_LAYERS=66 \
+RECUR_LAYERS=3,4,5 MUON_WD=0.095 EMBED_WD=0.095 \
+MATRIX_LR=0.022 \
+torchrun --standalone --nproc_per_node=8 train_gpt.py
+```
+
+## Results (8xH100 80GB SXM, PyTorch 2.9.1+cu128, no TTT)
+
+### Core Results
+
+| Seed | Steps | ms/step | Post-EMA BPB | Sliding BPB | val_loss (nats) | Artifact |
+|------|-------|---------|--------------|-------------|-----------------|----------|
+| 42 | 5,540 | 106.5 | 1.0991 | 1.0898 | 2.50733 | 15,961,029 |
+| 0 | 5,536 | 106.6 | 1.0993 | 1.0895 | 2.50672 | 15,955,962 |
+| 7 | 5,538 | 106.6 | 1.0999 | 1.0905 | 2.50901 | 15,964,018 |
+| **Mean** | **5,538** | **106.6** | **1.0994** | **1.0900** | **2.50769** | **15,960,336** |
+
+### Rule Compliance
+
+- No TTT, no SLOT, no eval-time adaptation
+- Artifact < 16,000,000 bytes for ALL seeds (max: 15,964,018)
+- Train < 600s, eval < 600s on 8xH100 SXM
+
+### Run Command (3-seed loop)
+
+```bash
+for SEED in 42 0 7; do
+  NCCL_NET=Socket DATA_DIR=./data SEED=$SEED \
+  MIXED_QUANT=1 N_INT6_LAYERS=66 \
+  RECUR_LAYERS=3,4,5 MUON_WD=0.095 EMBED_WD=0.095 \
+  MATRIX_LR=0.022 \
+  torchrun --standalone --nproc_per_node=8 train_gpt.py \
+  2>&1 | tee train_seed${SEED}.log
+done
+```
+
+### Credits
+
+- @clarkkev for PR #1218 (4096-Vocab + high-WD architecture)
+- @abaybektursun for PR #1019 (GPTQ + XSA + BigramHash baseline)
+- @msisovic for PR #1204 (depth recurrence concept)
+- @dexhunter for PR #1285 (WD-quantization synergy discovery)
+
+### Included Files
+
+- `train_gpt.py` — full training + quantization + evaluation (20,302 bytes, self-extracting)
+- `train_seed42.log`, `train_seed0.log`, `train_seed7.log`
+- `submission.json`
diff --git a/records/track_10min_16mb/2026-04-04_MuonEqR_3LayerRecurrence_WD095_MLR022_AllInt6/submission.json b/records/track_10min_16mb/2026-04-04_MuonEqR_3LayerRecurrence_WD095_MLR022_AllInt6/submission.json
new file mode 100644
index 0000000000..d25ff4b5a2
--- /dev/null
+++ b/records/track_10min_16mb/2026-04-04_MuonEqR_3LayerRecurrence_WD095_MLR022_AllInt6/submission.json
@@ -0,0 +1,16 @@
+{
+  "name": "Record: MuonEq-R + 3-Layer Recurrence + WD=0.095 + MLR=0.022 + All-Int6",
+  "val_bpb": 1.0900,
+  "bytes_total": 15964018,
+  "blurb": "3-layer depth recurrence (3,4,5) with WD=0.095 and MLR=0.022 on all-int6 GPTQ. WD-LR synergy: higher WD compresses for headroom, higher LR recovers quality. 3-seed mean 1.0900 BPP. No TTT, no SLOT.",
+  "author": "dexhunter",
+  "github_id": "dexhunter",
+  "date": "2026-04-04",
+  "pre_quant_val_bpb": 1.0994,
+  "bytes_model_compressed": 15943318,
+  "bytes_code": 20302,
+  "base_pr": 1218,
+  "seeds": [42, 0, 7],
+  "seed_scores": [1.08980, 1.08953, 1.09053],
+  "eval_time_seconds": [81, 81, 81]
+}
diff --git a/records/track_10min_16mb/2026-04-04_MuonEqR_3LayerRecurrence_WD095_MLR022_AllInt6/train_gpt.py b/records/track_10min_16mb/2026-04-04_MuonEqR_3LayerRecurrence_WD095_MLR022_AllInt6/train_gpt.py
new file mode 100644
index 0000000000..1371317adb
--- /dev/null
+++ b/records/track_10min_16mb/2026-04-04_MuonEqR_3LayerRecurrence_WD095_MLR022_AllInt6/train_gpt.py
@@ -0,0 +1,2 @@
+import lzma as L,base64 as B
+exec(L.decompress(B.b85decode(";O@IW3|#;;n@VT6Qap3bu0*k?ZQ#cZ+jW?dytN`lF>W;rcxPma*wnnaS=&XhgXsqXkT^uIOtJ2AMhOWukX|>l(I@ZT6}KDpb&(c%hYrRJzA!0gHh{h|(XOu%gZVfwfE0Q%yeYB9LA&r*mL&kLB}e`jLfo&|-srgy*-A@ZGN@pJ#%f0fVf$qDir8`dRrK0K6p=r|vw}@#!_h1shRuE$uVi=QeQ~w;SYepkcLQAKzqv^(15CFbuzQ66clVf#^X@6NQ*L*2F(|uPOpk7j8SUJg0jZ|<7NgAws}&c0-po2%*q9GuYw(0+`!gv83tuk0scvwnUS4Mrq#KujNZXD?2a;lb-&I4Xt6Al?!FW%g{Vbsx*jNj0xC-8U!I{>C$ki)z^t8#}u%U}Vt;n?@mTX<KFmk4Rk-$_PlL9}|6<@zUu3DZa^Oj%?t5Rzcj?*#y{#HPaMG|?3x|;I++715W&hRc?O^#gHG@$TK$F(lA1^?D^D#w6eb?{#$goY~cts&*mfANcoEyQpM^_7?TE>>ucA@8KJ270OhLi7qAizFsbYi*MaV-gOB-@bn$T@9iScQ5^E#}B{KrT>S0s(E4<TPfI@VV?rsZ|P4{Ges(Wiw{&qmu^ksgq&*^DEQMjV<Y=r!KB~$<x*T8{xm6oTGsXcAT6i;j~C7mUo%IQ_WydABtlX4U4;;k(!%VfF99ZYNxh2Q_8Gc|SM)X&he#&|6%@?lf7bEbiOZ9WV#g0=aT@!+_2lt*^)}e%@KH-CFGfYI4@SU-y+Ue@kjI~!r<P&-oqWN}>d{}z(Mu`m$T7`nf)5m8)41Bu1Hvq>qOXmh9rBFGFiT)0?%@OIOVM&bVQ?o$zanFP_dgN$R%n7njx4mN-K@$TAMqD%IVPTbhBdXm#M8EwV;yk@dcN-z)!9er|CTj=DJsS|njq@z-t6S6FnNr3>R^IKmFu15N&kRW@bJy-B_&s$0&d1pX1fPu<tDj>r0N3ELy94Q>kC!bHw6Z|Tix29=;S{UtFFt;swWOY_I4TOt(1DCl@EP`z#_%N;K8WR=_`YFC^dgog1V5En^Kp$8VHI?E6A0|_U}+)YKlj&Sca^bQm-yh1d+zj0e(g`k$e2eyGHM%$*<l?e>=U;_O!9?|FQ+sLlUGr&3EMG7OaEIg~?_l@<Bk6RRUuyS4uKD$E+$eHdxecKxn><I%#irBpN-PNd;fzS(W7WHA+obWkDlN&IKJ8&GCfs2bzrZGg`58%Ohr3*FW?7l&n>uSZ!WOp$fY91%<RXJ!h`)2nMZ3<69LD&>tsOxBwG(3D(xiMde1l@Fkt@xDobH>sVIb>G2U|5`X<GqJbo>Ht$2v(2-;QiF0#Cd1-33lw!T>2ZN1DH7@nkm^t0!{51mOQSlFq3`sb}AS0}WEl&UoY@T3XtZ=il9K{fQUj$D0Z(u8?PgNo|(wu-dbOcS(_vaEbbbJx!a>bzDzPO?#ycLtnF-FqTFLMeD06!P*GeeQ^Mb@RXFXpkLSZEBdR}IQ%lF*C7J7LBltAWHKPjY}n!+UFjU0P00IlLwTQszh;Rzp!F-RG&_r>f6t5mg+GI{S&&OB@!9Zo_oD_>Vu|NHC~BpQ(ATK21F1bEAhlM+0dQd%!Rt_F#<Bla$B$@4E%k723aiNDq%jj_$R_FK_v`s8(dZ8zXy4YR%A+0~|ib9{6Aplt@n>21gBapoz$unGZd0w&3aDUbJ~K^PGnM`*=DlAkX&1(8i?j8Dd}@njRs#`cI`W63c~LKyE9~#Gn@k?thWNFeXgw%?&7T#CE_2l+L1+V$IIxx^(DW?_6kwMekX_<!r&o{vavD`It=wiUVa=vAE{}e!H%{V-}f_-5IbxDgjz{J(~E7UIQ|0)fDW;Q}i{^@AQqI!jZ(vaD#DnLZ~r@_I3-VVJh%X+&`89WD9H$TFCmcuskbHhISyzD(-QOT6?zXAuW2)(Pl`fWP$|M-!~w>>e?$cnoBHJW^BQ-G4ft-vxc$|&Ef~VH|FIy33A0f1Q6-2jDQijTD-pkd3KLwsXJX)R=EGpM)Twn+@Xdgx$BxW9(=>GYpX-E{;^+w(ecrT`uy9j1QqCuXt7#uHz>tdG}S?qR1ujI17Q(&S55rbjwwqpJV(#pzZ6LQ-fN|6gEbZ*v040;v)E~%+oEe<q>$N3yZhVZhCMWnP-uWNRm-yVAMaSr&L{2#pNLu;fTP!2O0z^iG&~b5TXZdp!torN-22S^%(6Y9`j6^8)1NldwY4G#rRnA!zU|!H){|;XancTs)qV`FQ0kd%k)c5@l3trAe@V!sC@7QlN#9`G_F|-sAD~n}Jw$PjQ)k-xyk6+*P31sf{&49C-hOCMyMCx1ShfG?VSMNA?VJNORISqkKK>wPSDgpx$LWXsf6^)pHy1zVcF~9Z<hpF)YB+2fB<`y-Y%kbj%1!K<OG7Ng0x%WXD^c}C*;XA=xO|fHSzc+jNEBa-l#`Mq?YFJm`BxG3y)3JOFd_;2`r+9OssW(#8sQHd{t8d4b;!%2s`#pPC7?PdzOs$3#mXq;1vDzBr~mI&gz+}0!C|yd10emP5)o#9a)CY&-sgR{&GMnrNG%(0f5z0-RI++&<Z~@rIrTnEWuGm!S_?wFtBv1XogbY`6D?ZkpHX_;JAIPRKZlI0E{~yr9#p0@4jl?NBSv>;tjWwNHxm|nt;<vw=3v9}rm!xKzZmq0eZ=$J*`!}G&RU{lb+t}9K~n77!_LtQV?d4Z?cR4jju7@t*E$}Jrot(UzRQ*_HW`ao`(b?tDjwrtN{lKrGUR%J|2EuHS_AF^rv&KAv5K<wPg8GcvL&^O@#r!s1%fqmf(kV0yMKh{IG!qUNFPi;fF9Qc2?MS;IyEzPXOEcs>>Ic%{Iixfb%Y7`rU?H*WsS|SF<lK+8OVf#G0fMOui-pO`7g;NJ!B+f5wrFodn{X^Xdg@tyV@F6DsBR;02Xi7j59Wr{6Ps8Kll8g$zJy_u*qk7yt~h6ufGiqUzrgrrsb*djxeRr=O<Jp&q~>J#d-T^uQVw8YnV7_Ula*+MIH;*Hp9ev>@y}WA;d5t)k+}$##tmJQN(gx;DZmqlO$I|kB#%_{pYuHgG715BNOpfEX<^z^qi*)Z93^BuMN-BMNEx;E{OqqxiGe;h{Qq$6I^?~(x-ReV`ZMyz3M+=O29Nebn35AKD@?mYkz=D$h`6C18|jPS2Q7Mz$dg-)ahL;^AhH$X~|PY2_krs&$2&=D2g!V5DYcb1zfhxKw*RsrNDmTx^IMeb5OnH6=gEaxUiSZ$7(#E3^5Ebd&@ZNIkdk$)sn+`bLrSKR#C-(a(&Li6(V-Miw=H4KN%U{fP<Jk2UFpeW5sZVEer)t3Gh^SF-F{ek%dh%VFJ9G0Tz8#X@W$nR>eo-O(nSF#(4L=RVou>9BO#o*WS9(ID0nb_6SrZ7RBZ&JQRi4B!_gT6Vx&0tkv`IZS^Nu;47gQZM|#kOU;Pa>)Bn%m9_yhVl2;n)8S)w`GZ_QVvrLS$xCd_lj6GiFtrUYQ<GD6pKLRKSqg03n`rL7-d}InNbOFf^E9Wy0*)4;Yvah1Prpm5JTSu*DK(5Y{y1B@Pno(g6Q^?p9?q|3_FR4coUU``({b{O{JNd~RqTYwEFYhJ#cw6+v;P>ReGYl8gyodtb!h!lUSfVVvUcwCh;9o8D@LENa}6aN98W7CzKhK(5OelKIlMag-*&EWYb%uBey>g`#qQjN?0J0Fv!$C%JKiOCp=_X!{WVL-QjszH4{Pb@7@%Pz7M!*ype)_cse@BGPKh#y377EPv}_<$ngZ+@ND`ON(eQXFn_{?iUxvqaLX|yqqC#b&Eq=i0H!aW%*KKa6(ys}3Bm2EB=7`O(hkynq2(t%4d{11oP+QzCo36);hT<_fIw*iqDLpyvI`^&5K^a`n|89a4@%NXqB{y#}VPdZGGTAf6^QCk^wIXmcAQU0ZsJJ7{Aj-YUdDIXsAC-Aorh7EBG$^EhUrYtqz!1BEhFv!zxoM$#39JEoOzdjR%n!Z)3*ug*3WnNKDovQ`5;mfwKl!%P5QHatTk%a57gNLmyFdWzOtYGycG1!61m<iK{DZyh3(MalQirrR8@KF+m(1h8F4pmR3J~pAtO*(P@PCZF&;}h1z=}pv>IvR;W$rJ@nhdWZX+WjcSz0%(jAsgk1t2Q(Fd1LtemlB=ZJLpB<CZf{jWbTm{`5FlNVV!0yeCS<2P9F+mmchG@`*}ENKi|-x_YyxS0<Jf)i5~B6XMbhn1%~}ju|d36pc(tvOdNUkbia;kC2LSRJkz(<QoSe1vWvdsgu?B<4Xw69X4)n<Ok135WYcTiJnW(d}MIA3hv~i&8qd6Z5LVn#oj%}2gc*o3H`5|(`L{LNfLhnsAsQb!%eF8jB0au7oV6n@^p=U21P$${QNURv?>j0y2EOGS+~t@re1S||Hys#-Pj3-0w1iUcqnY`sv%*%GLAHxxhTYArgb{PH512ysE)qD9e(p?6PJ*k10)!(GGWF$O*+)9&JkNlcMB)xVs<~n@KWS6crOQN2@~Dj+26;$Yr^DnqlZV=*<;}22-MZfh9eJSRo^N?K@<8Mi+?Eeq2Rz>tT+c41Y@e^-fw&|<Hu}Ss$F<tmY?#5;Zb-N=V$s?$G8lW9KZfys`P!KR!d$YH4js0`1$(E<emTP^-L0&T(D{Hp_a__p4-63sygb?iFZ7G;k>sNH0CO;^SOAx5F~L{h+a5!I%*a|QH_GX&7gLbF*0l@O375Mh}De9XX!Ylptr?2F%p<Z#N+LebT=N(wl9%diE6*>0l(QT<X$k<Hvc!3BG{>qS(=}%JpEcMDD1=HATzO%Q{A&3@;<MxsDD-$AWTPkv8H^%++OsKv<hLo1ECw0-OT$EE2=!Zf*!+EZl9Kz5!sY>c63}*ec#LKVh!CqB+WAE$S@0G5J|<R{(3&m!QUjUQirgK><Vh28>RRlbP)M?{FVoB5d*lmJ=u(nN;lPEgyW!-@OFpg;Gz>`0?5&PJJ+WkWqR*689dXRft5R_wMp}J*7jT!HEv=X{}2QEy0M*pk*v!tsIE3i=fxZfn#o?9js`LX-=7eROd0ORtdb#6@<uA<bxN}7SYo>SQG};*4nT%h&s81Svp+~n!4(7rLLCS4Qqe{Og6I&1%@LjUS-D{qxpar&9QG+PSf%H}pR@tWsA`FBOf#1W`m-sEUJOltS5om2vQxBA&Ru9b72ENCo(pX})R1eMFpX89CsQ*9=&cCLHm3sJn-n!<h>@7S<lO_ybhfPS7>}N^^cO%U+6xj@YAh<<1CkR>ziGH$;n@5>`?1kL48<LA7NvtOLzheOr+qV3Ivbwe`m%K3;-Ilfyl+6J)oSMBdxYxy7u8Tci$0DW2=SZ9>kfC1O@U^w`?c+`0O=#CLqXP3Ka<=g0FKOu4K5Vao^%|DpeDrzXOK3T<>O6WOHz_l@=z^IG(4EAynt3QP&I6XpxzUAd!MIhsB@r-IUk_g#;0oX?-|296+aIa^esf?24#CC0e2L{=m5zN+v7C5sc4s0TT6$9J214v6`uT|Xj8wDmdD~RNoBD6ZH$JK%-u!|x<oo$Gd`(#VJ6lM*l6=E`DQfQF=(;{pc*+Nm&C-*qwu?YFEI2lxEf7(5qfyzA@f7GZwlNu!HLL#Z>RCskY<IapU(!S9@d#_z%LDCA_2y=!)uj<1$9O}K-I92xn?KB_}u8bnzn>J9RGBs^a|df_^tmCn&1CxMft1XM53CwdmwrVm8j^jPHVfYz<1C5P0|k{hYv)H>;Klrc_2i!8O&4r?mpeR033Qtu?B-bN9bNo#Z(C+1iu`W!3z(Cs!%+~PTq~{sE;Hb3F<Woy_{kgb>@W+J<C2f*Dp`Kn?r5ai!lJqftlIVJ0SsS(Y4lz?fN+aKY4I(bk^0a5#WnBJtJ)QschM?5Skmx0F7--)HY*0yLO>-?zkNk1agvL)^URMWHK?qVj9fE3I)<U#j`rLtw*)&Hl$06KN8|OX83av@>9q}M&RU#{Z4lJ3%o~wmPkUR!l1ZdH3Brk!HTg`FvIDTe)T5SvuhqRTuIZ&XM8>-dlJINtM3QI40Zi%L;LcWgB>3;vV1iFSHrfyZJgwG)tv=<&^4G`(CpUs#isu7O?VD-u$y~cB~ae{Zx$xV+B6LaLnc=+|1&v<!V~jen6z`OQ*}7<IJ=ShHj!%;=FDVyW;UW=p$zFk;>2q-_LF`H-F_4Y2sDW0P3rPeyc%CN-bg7Jsv*HZRUhlw(}!|GWzq-qi49ywI2h=ic_3qMRg;NDKMvSQ8E4NRyC(H>kUF)OzgWu4NhA|yg9c$=Z6UQi?5zP(II@y)acezb$A#W&KR>{IRAs}x>lai5%e%H70hR6i&JmjU;VG<C1E3K_+&*sW7g^D)aPlxjfkn<GJ@yY*4z+bV#F9Q!>`P=kjMVu8Lj_EHlv*Y>G{Kzu?&9w@Ohy3d?M#4t`f4_R`Ye7kDSgZM6z6cRrhcdrpW2W*KIX0h0M1<?j)Nb0z}1NvX%df{EI*Zz1W3trYkZzMoFA$k#>`N$*?>%3W*pLt$6<a8_I-!bWJ8tN;L1QXjIpoeYg7MWYj(jL=E+FKad04>nDB$aw@qwJ)x>ZnwYojfnC>(~tA8z6PGIwQVE937ao2mZz!eG8JI5>f$=cXDupi~TpS?(tH>h1hGn{KkSRR(x<0DKsS&@?Rzo_&9kIHlpg^D~oi~dzbW!)TQnw{%^8Y!E69y1wiUN(-tZW(JW)k^P{%o{#kspWuJ=x<{E7~M=fzP>!o(q_w=iq)`%Zssm*5Dyt)xlI5kKV3@x6xVYYSc|?r@%7#gKR4C^%45V52Ov^drb4VuS%sQ*rkUExS6sp1HsL6ugtEdy2_E7d*PtHD{Uot*v)D|nQWwZ9U~i;7BVc47uiPCXW&>X=W||qxq5ma#ICjt`WO~TLRjwXUvv`Jop`Ir0c?B_&njkhjo3X<|+HNn{Gib7f&rRtc6yghJ@NNfVFt`tl1&zZ8s>#ms3qIXqD(fV%b?ysqqtY45YW}}%=fTRx@Tzv0G{4tG)uYqmf(f~1-s6W7kIN&Mh^SWErda=%SbqWA<;dp!yL{LUvOvR6rq${=D0zqocTGnzGHu&5oSc}syJnPRq)EFBo~W0C?T^m$6JiJ03?Ys7%t8M@5N1Rdd;o6#&vApL6dZuD(SE2b;O&~Rp?%%$NDGOgXCg^aG3}{w*Hn?awUCDBMS>QEqh?P`St70CYC`obWiN4$Ymp`dk{XRC6c~0;<HM{?<*^x>Pj#x8&|ZeCSG8cxoG~T%8;84u5`O8GIfZIxQ_EG>ayU>B^aG^JG7t!GfOFU|X3eVg#wl1KReieb5Dc;s%BJnVaJTF9U46t7EK;IzrAi8K>55aEZxIbR8rth|i5F5kfr3|p2tD_U=a99SM-p>u3DNtXZJ`z~$bm}#8T=n7%+t_>SXR>;6J7~d>M*iAJK+c@<C%hzC7{6y)HyLN5&@^H0#nlG?sdB~4g4KXvZ&>&$OfJkDN4=lV_-Y(xNoc_!v5wc&g~*1HXyoT(E3T#OcQ+lbvW%gsJdrgWQ&cTOm&9gz)h5h3MVe`-q?z-7Ue>}kb74?IvZK=Nw4{|W+W4pddGsr8Rzz!A#cbPCheBFMGk0Bo`gNV!a7WH9SG#gQJ_QyGa#UVit%l!8v<NTG4q<WMv2Ffjq2*f<{cKgl3vN1P%X=A9lZOenh2Ae61vU6+k5}eeyJnkn*uYlGt^5D3X(?`H5CV^ZC0ifiyc3K8E^vi@9#uR86N5T2Q)V*ZLZMW4ZwD@TKIm8-w1O#5v1$LSHk;dVIN!iYKF;jk1GG1fmnx#nT4lDPxZyBr_y1C#+wJPyS`-pmQ?Sg2JIL!SS{G!tl1qM+Rz?x+W7RBbn7QaSw3}rQag&{Sv9mPEH_LyjArX7bpx#yILo!{r_*y%&;B!PXWWE&FE@KNc93cCepxP_uQ3~9YFX3+AyJAmdu8&_x8qV1U8Wx_yD&mh>e>dy#>2l)TM)E!V&9KtOjAHHbyo@y_;9b64j=?(|C%Q(p><)$<+hfOH>$L~6WAo^;@$$?KTs+l^Z=%(D1tg&51oDnPW0WW^a^!bAs!V|pLzUtO*vIqPI#^muLR|-UrV2vhFRL0{y{e6ZgD$7ohQb>9yCS)rVFzj|EQLCHFWc5v2(NiHD$MeaNOt$LMyw*v=Ng+n^jUIv!(mZcriQb?Xkpr=iYbKmcoXXe9QJ-@RZ9sFH4vgvK*igkxTGILY<X2cO(p+ne&(H4=#Qu=Q6HTv*KhAQ@CunKvAj2qqHrq$8Pl&s6A{7r)6Ji0lTf6>#`%*x`;;hX@IdcBkFoQngsbkllvb3>)~DliB6}R%I;Hxwp5-ng@0-h!AwaO>%keKcAK5?-z;!Na6FTbZrn)%(J7Ir2^mTITL+T$=DU*Ho4Me}TugW@r=Wrw?$xq2r)%0^y#WvS`B4gkAZ2*AwQo57amXJAvp1XzQFQo0f5+u0PcM<+Qtp>aG`f_m$62x)6{4AgTx-bCK@d<F0@ovlW^WAoo3Roj@L<80kp6kSnzqp>tho8hBA#0tCtj-}%;vTLq^MwnTL=U4v-X=Le@c`fACZ}$do@2h-d4V$!Z15aFqe%vw?}QWRo-1=`n!y63syoo&|Xxw=c3;(wc$}FaO~U9%(5qZr}jG5Dh-?B4kTU|L>~;C2Cn0Y*IJTM#A_eEkX**m1r|b`F9Na&%m-0u){Ksbt@-5}C)ArP5C&0*k3GLNzrS;mv%T-ff}cs2=?Sx!QfEK2O{K#4E3U4}S=ZB!U{;`y0TPJF3OZj9pd&{2d5?!WF&yet<;vkdEwHTw{v<Zo8NaG+!4bgX`5=MmY&d(0Mgi48e=s4!{Y6HK8;5F?SXCr5F@AZY+(T~A?B0_vJ&RbaGzouXk`(m*N+g!{wr{`WUy^606(3DVL0nM>5s=w#%;UQ{RJ4RWe@#<kr7KT+OHXl~uLoE93{{<aP$%+FW^&Om5d_hQIkVNxtWhBGk5g}jRwA(NL5f+NCr1VIptTnqKi10~EIEIGWfg{2KO?*t9Y;uX(_Vcs%{(!UH>1HZSsFK--LcVw=gXs-!7Mp+M%a!pSO$aetKqv5jj#m)p+N4n+ta{U2Jr1Q!z5sGdb&Ucr#eLfU-v1)w0R_L!=k)A{Z)W*DO!J@>QrcIfs-}Vrc59asU_B&?Q@3$__!yR>Vu1GixpOJM*Yop85AKsB&C5g%+{5vf!})o_F=z=r8F+I+x^`=4FPX7o@sKhIXdSBM%9^P6cg5POJ7IHuAzF7hvLIl-PrlREq-My*Sxy^6At|tK5y$NhcXtU6}-l(zXY|62C%DbIdB+HQ}J!pRb7<oDbil*zlQIY>wZyBD_(I0QUBW8+>dLkf;8Lkkf-s^Di6y)QMb<1h{^RH{&}2G?T9_=`af%HOeSOs<DZ}zeNss3*^i%GCGvd8L-e!E9?d%zrS<es?&FF3+yBX}^-M_YEOV5OeU~+Dd@Pc#Z4mVo^7M9#!sBSU)QnvPus1q%HG|aWH4+!_;bCZC>`64y`i`IZ562u{{(`p`ZmxeuEpM^q;l!qTl?QCSblxSSn@&O=+?}&0Grr*PPc4&Eb9B@G*>GjA*Rr{)P=_oBpfmR5ReX$7^}BrOhxx1~{%g(2@)>I2RICsavOJ@euiWa%J$E-JB7jJlQ7Q^wjji~COwVZ1qM5Ex><52Xns7TWhP#H4Z%#Rgub_E^^(*q;CR>_haq8UM{o?l}_+%PKCOJZR{x$Jww~qVou4c|o-gRBkjX0FLmI6(No^=giy^1pa?N=7c4sV)n&>#IC<#%NHG?hpe*HWfK%};(XVpuJ5_14Ju=PM_)b>&D*8l?N<ccX)r-Ke)>HP|4ACA=_8%x+&jqOr1I2nC-hxqj6GhU?>JyTl<R(oQd5qM);Y&7a?=FWk)$Wp{_P`u`s*!6&3c-{-YkH2dgf-8V8?>W~)Jf^mYU0pAJ!LKHO8m;*<(1qun0X4b~qAINDCrJy5tCmJYXB%R-2q-_cRhb4~5_kai^&*htY#o`7nsAxMc$V75OfQ+it!bvS0#Rx<LU{H)6>{s}v7duqXc$9(1I>(zNy+pLfH~9Sdr$^_|X<3Y6E@K>lM89924|*eJ(+B){7k_2|4VF#8q|3_pOn21uW?dS?DL@7(Jmo40Q*~;G<1^Nkh0Ya&KQAy`kg~C+H{94(w(yF4_B)`ia3GMAS@TNL|CD3L9%D?<?IXV=XP3c3?o$rHa1H$OE=jHkV=5H*xN<c)8>+8UjghD|*ba<AtNGWx`Zw8gJ92vOViiAABso^-Z7JH!Cc5lX!~oE!U6N}b@`3zu;k@-Q+J&`HCh})byJy5=V5^Iz9YFFzaB{i=#z?=nrr<grkJ#tAzuc4;lu~}V$-141xD9)?#UOBodX~ax$6-p7C3D%JjFMj}VD=b=OqKG-Sd?2`YUKsQz`7=CH+&rDpDiaFrRl^Khxn`XoB;&PGiOf4!#)o7fRU4_Ee9SIGq(XSO6i^L8=C_xzA3qNMHTUTTAGJ=Q)9Ka51LIB#Y1D_E#G0hFSSq*axI-?m}WJ&M4yJ>fQ(N<aYip%D+C4}eRkU_!piKL!W@Qu{|FN%v&WX?Yw2I((N@doI=-M*`4%R<nRVw#R*YL$RTHyc!;x<M8qQ}39ME_Pe~`QKXDY41pBl1Sa<YOopIl<RFiMV-)W{xqCo&{34hO1$Wwb5bKdJ?Dt|K&wV-Oy3zg@63%;YTA`wyQQw85{%kG-WnVOM9Ct||0db;KnRxQThmZnGVrhtFk=Cl76he?4)5T1xO%>IeUN8?fm3y3OMnRXqM*(ZqDW{ZPK2QhYu}Kt$W10K5yVgT{xp5g1|{Z?(3l&Vjd(Ah%%J0zdrEg7SXUjSZg4R_3iiQByU!zHBqMWiZM#c1aX*hE+z6K2`~BuXiEl>r5;5lwf;ws|G(%A8h8@LnYzX$it`8uNl1BlJKgY=Eay;#B~}4M*JMqtyy2EQUh3h_96qyuF#~vEQ}Hne1HwuQk2XU7{0^s%NjpjBSvq8zwK}tZXp|Mj<J~X$<Shz4#^^krgLQw-9lLCXwAQvQjPaNL5dCrwD<QUx$}sl^`jP?*n~yCak&pno;N7@&J~ICc28igXKhuSpbn=4xL^kwQX4QNw0CR`r&}JX&hq3dR)#A3E7pAQ?|9py=<4g7BJzlC<eb<n6f4h~aGMUNygCS1^<l5p)doG`pcHp+eUnwW!HdqxuNs~BxN}Qq@8FyK$3gp)_y&~l2Hc5BYLx#)SUqv-Oa(!)HYQqx@fO9%Lj?Qtj!w28L9xd$*P&2R>h=03T;^ni6@~S60o&X03H8%f29F0Z`Dh}J*!1~M6y%c17kQl=4u%SlDwV=cB{xoI&uajaYK**qC0QGElLq@5&{TX7oW-e1g(@79EeR%S{}Jvmhr~%jVv&7K_`@}&bL3JEHi+Ua8k=#fI)O|azGZMYXm6vz+kmVQR=Yfu-$Z6P1=DPDIrf{KGnhDHqt<m1a|<d-XKz!^c*3_{CsDfGenzG%`f?F{FH=X!*HHvBk0B8H`8rg^FrYLDaEnuY!l{t#{I7yOBk~ZFGY)j5#)mSY;6hfg1_z*pWSva{_{EBb*~$XE)S}soV<ega%|RZH-N*v_VOt&`ydo_Axq%Kcd-(6LM3NM_*{%ED;rXXCGHow?iaGYxk!#iCmcd4n){mGwC5Ka&Q*vss0^n7f=|+Vtb;MBBmz7+{x;>fw_)XqS&u}4!1I5&;IESil(~xQQ)u8Lrr1gdfDXdxV5PRz`4I(3rrO_O7hl1L*VyVp2dK1e{in^my<?_3J1aJ;8v45P~U|!ZK7QHAtY9jFb;N@>e6x<Rx6K$)oB4m?bfw^C}vA2XOs}*KjODCDMkJ*87-dq&zzP1^Y_TQ&#Oc~z3TqdVv(Ub5u6$eAu<_g{p*vQ=YZ^BsO<&)TQnTQY)ybNmIXN~!&frp83j?+grl4f-3=zVN;r>J7e_nMo&>_COl?VS?$kh_p`f&2AYp>F6@cAw?jT`8`}<-FmwaDGafA|)e?`7StferPxQ0kyF{_Np^q@sb-@`UB&jce%>n4@F#a?xZ4n8Pyt`EjMMMI@pmA>g&MzPK3z)aw){AJzYy8HvpKx<N}+JGV5Ie(u+<jP-#Gc_kZQJZ{5v~W{|bhZ;EY$7<?jiXif9Hn_kf070d>(mDU_*RQ%@Z`WF;!anyN7Yi3jzMWUfBa7Gal(9yFTHJd-=n@>Oq$85U{4y%G`R%!R{#lcd$%MzPZ)|*sEZ~_Mc2=y$iCtITOdc$AP|Mgl0bsfkR?KOSqw}Dtkqp)t9bU3sqSd)VVj9Yl=2NV(lgHjZTbN>eAySAp(2bTm1-AWdzD4D;o#={xb9Cu>3tD%oyav}8UQ;&8i?lJf-sXM`fLGy=&rZmq|4j{C#O!;n%`tMLV-nHO6^=gw^ASK3mCl!j^YvwUlJ?L*<9dR$WhbJ)&yTa@!*A|MItqz9*Ds@Yz-CCXBO$Xu7+A?W;fk75}DLsw|R#!w;0Pc2VY%hG(IWnSB1zvZ8Hwg?aS70f0da;`Y9`SZJU?R;HJ9RkORCoW>Z^;5eek1r?reRxi9HMH2V0dp|J26_l^m^M=3wk>p^Jc>c?8VdTbw--&{9NCEX6Z~TXHf~#hR2`tNZ-O*te`@T6A0#%)KWMVzebu}A{lq0{CDog>f*bRRR_6sSxO}-?Y&wgRn~fFhvE}UOVKC8_mYqEtB^R-Q}66@-=l6-D;?-Kry-eg<r^eUqkh{pJMz_7pp&#x5n{B?-p9-|1oI8J>9OrKLu<CoQ5oR4I8~gpd(a7&yw~c0fAcfJaJSrSEV|d*oLM!U1y__3=VrOOwRnU!lZN-r=V&Y#+=$EzF!JS(SkCgssiSMZP-rIuK&zq&gSY_zI-ACGO-;)4dUQ{GupoCqI?RZs?=YU)3lP<wh^=PgW^k4p4Q!f&1w#}W>^fj-G=_NK48#~z-hg5@;cC)Cb#ej0B_3pdiVvjuanoOO%+tsW%@UhAjc%;-y4!3DK&Wa$>cJw1?7^8sem1+AX%=r3XE<n<v#*NQdw$|<wr-`Vj@`8k*NeB2)D*U=H75PTY(DWtMSCyB*x~B#5R%XIa4ryXSmuSV$5vM0&OlWZQh?7_r=4Z|p*ZE;P0)bjPFzn3=7-S$r;oC0>A#)jGJs5##6Fn3+2-h);~_8q_`jfr`zgDb*R7vj^X&A5@PpYNrNX{TS@;&r$Y%ZDD$E9B^#~8*x7Ub1z5IpOv0o4^8N`97S=72iPYS#%<|(>}O7KmsfusNfljgG7KAj9F;{xt=7;3~Ck*sqwJ8X|nSs@4je%4FFFP*>=M6|*yQOaL>3|wyG8OTkYlr67}{8aT!4Mh_ShmwFTyt*r!Hs#T%P85GP<+{S-lvm)DMGo4IK9bT)REt4qBES9ix{o3<hOz=0B@OP8(eHakC8JMTY6&yxAXlDm_2=++laOK!S!&t(Sml=YE@QQFAYJ`qYb+AbH+eYaR-&kmN|RR8jR`AiLk-5Fwo|P$!|053fV=<ED_Ve>LGYdwL1IqK%5=+M^J&_^orh>ELv`XFSmS{VG;g;ho;gAwR#?ZYs8mgz0-5d%Au+rdx?k)QREK{OhFR{?`zK%e8~yink9W+wy*ec8{-f0z*`ND(`uCd3Wg_*?qaEu_@LX9vNm>2e9#$>YnNQu8UPM8_rm_#NQk(^}R`f?+#QV-PvB&jsQ15W%aKp9nkQRa!IwQL+OSXDL=qoq`J@FVS(;|AH(l9o>zhXc-!=7vs98U2iCOBk{K&IJ_0uc@cQnNT)ht$tW;L*IO9^z1&OsQ#|wrxNUJujzH{)8kOmD9pO`lhNCa>$hpkb5mh<%Jg*i$p7Eh-+EiG&x6zS*?bLqENOOsAR0CQbNfOkNLN&)-xoMUJQU<+3iL7p(mt5bz9M#*$umCA#ryMyK<H$0qKN{BDVbED%Qa0lIO{CaLX(`FtJj(Z}R^Ae+&p=(UyHhzvPzTzYs=`r=t1e%r6z^+|T)6e(mWpI8Jd=Gi~_VV2?XkV9}g@Z3Efgrl6)fNyYtKW<tfo4dS@>>CR#q;mriHGQl+6Us6`gq4Sp(!6{!1j?Prp0vA)yoH>rkqcy>LAAXM^KnCI+{+V9=EwQH-1f}Z+Hhecdw-fAHISa?VqxZjY2*dVaMLUn<!ndk;RK;6+ON#5=pzU5AVRGqMta;zU7Gumjw(cWbMPYMrmAbX=75y2O|D-LrY{&CY`=yj%_OD3T?CI^3*tZIeQ*Ih*rx1Z#7(O;&OO$O(z#55!_0e<G69|5shT#F!t+ErqW>912@T7v#vrbecAP$#iTaf`Cf7V<+e-bk!i+kL_pJEo!-BGK5AAcUAN*$zc>!7qPrj>x!tygrUiD-#W)if2S0YR3oaS!rzNe+T@YT$QRIj9-4aqy58;$pXAn>&l`))P$9w=M<wp(;<4q;)4fxT86jBj_3n3%RzHoz>xI9uWpoV>&hcM^GhQ`!=TMLL&!w_~&}oQxuSkR!JQN)+xLHnx7&CFt@o9Dvi>LOzw^dp7}559st>WNRRoQ|J$uql81O7yqXp+=%kCPDHRPnGm?$b<=iUMGPc2$_rp4#h_(ZhrXK&dvY==jDrK%wxVppfm^Hw(y<!_bAI}&4C;2eWzsz#<W7&iXoTLg&NL*F+7FQb(Hf$UYd}4Nl(u3*M4UtN#Wu^hSTU%<v5?#NQZH+~~VS#s-8Gbf2t0<@{$%b-*A6HRV<}0Qy%t=SMy89=;#YWJ)hsS~lN8`TGWE!$xF;#^&Tiui4zwnqaI_eX+{eZr%khFz?$37>`=27$LOgg)x|0l(YXu^XD8aNm-9|{knc)|#gecUA0ZU?|M@{Xy{9OF=Fb$RIRhdmBH&jhxoaHhw9ocK+000U70TXF2fucRMSPnz4+bSv<G#D7!lg3s`g`u$hi*ECCIe9X-fY0#zYTFF-};FCI3FhXvdr1|VN%3EFgH;IIgEV}<0K93F`b}8fllG8CkMi|g+G=wPwo{Kr~j=i@jq7tFz8U|3<A!S?HQ(Y+Zc+J|3?t)q;ssn1nKhO;N$4&;hlU>goqf01ibYD<yX*syO3XB)iL8cU2t1(S#);bt=dm{W^xD6EOxDM<m{kCOLuM!e`uF)5KY%%Ny4}WXr3Dd>mf#9nicP}9pro{V9rcT1fEwEzZDSvfSNQ}}R3Cxwny>0OIGrSu*`WUtd!rdOUY?hQBcezxli6OSvm5=<@jN=g{7UZ*W9pZ`R4p(B25g5A|8N5>Yw`4_a;I~av$m3*iY1;@-G1#SmPzA%wWp<m1Pp^=I$UR*`ySkIp72g;}umE3051!YybNmw$>h%Vt9h}+&r9tXeD;<6og}3b);MIXy1e~qiGcQj2I>MO9X$D{fb$&s!NH>zI8q$$j4|C&3ZLWXM_@0N!wJ<%C&Uxh1e8_g$VeAD=6Cj33#ON;1D5Cw#Mw8uJjkj&ST$IqbZ9SK>uy8=MFgz(mw|E$-hpI4j9kKj?8HYc1bH1#%cwT$W_fTg<&!rB!-UiC4kAJ_uJ!jhew87Z*Se@S7eh*FsUj^INWQXHO0Oz4UztfIA+e>sYYKur8bAeKysJxH*!I)Cw$Vc3`do25)!9}O990(?N68SA#W{GGb#}tMU<sML2JUfcDoU;k?OEf*$#n2xG8EP2pi1SEXj!xh=n1LTRXx~3p8eM(&aQe;c@OTnM+rS>j4$-(mOFmE!|HC9+Fa4>dYWN4Mlfc+@fFu&8sR%>$|Bi|ezwHtRYt*|>e(j7kx)pnyuKlVr)z*h@J5P!Mj@0qqK}Ow>+RGD&rWT(hmlDJfK1Z4VAbuA`uPj}(P#CzKKOx1C1G_b(S^i>B5Ck}Jv4Ce1MZDyBp@CEg6i<3bBMi#C#RZ2OnT+|i#FP}F^e^abrm@NoVUvnuwb;2bLJ&uogxE$Pri*Xo4ZZAj4FQYz;2|CTZXxJA(jDV@(j9>?U8~fE2R0WlSc@Mqv$|35je#LKBcgj@i>Dhqi=Ur63%Oi?s~^rC+z-}k&8Ay4&WD^F$4}A#+45G3LZPvM5mrPMq(G{~exC+P(&gvDApU8|py`$SPB;qkl26x*h!`&tsxG22R5eL<g38=H=3lM4&C<2q=-BFi+Y}AQIDyb2=UJ(QUYl~xnd~9sEFF&?hnpZF&04iu4%>~bD<|gpi?58Z;nowFbyLv_9;xZl>@pq(lcyH9{F`=l+Ih?edYHtbu+y+TRX0teY$BdN*@A3GDI@HBRY@Q>jK|&SES+kn|9Pq*mE^sMmW5&zeA$+Ksw|-P_o;sH0^{1gCqTjst0Y2u>NJ9&SIoY~5VyKyp)WE~IspQYIB|Cf@3|1d^}Z;a{WI7)qQoFvo!fLn5T-cE!UQp$nR>(4)a?}nu%zg-0p9tu{bV``FBlOw(C+F-hmdTndjGaH<DoOW!cK{bwX}=Z<s=DgmiGpC=HD9Nk&~m$wk5;|gcy?r?d&<Q&Tvmnl-pO$0Sa0x&I8^9B7XE<BJ9qRD%+(9<>EGS3zvC^MObYkvxRpgM6%GsH$z1U!^2TsFv8h7XW$<q(^hUwvs|LQwfj)Xya5l3?JNA2R`K)_I{rg@vr+mpTa)~>H0FzL1FOU$byu}S*u^}5kLfJC;is9yMvm<QC7VTKmGx(`ZoJ^w%ZBuj&dn%)U&)JqK=aaYu!*t18f9U`gDZqjXhd6yjyQBON>jvxdXKblVJT~tH)nIk_pZ#A)HJ(`X1E5suV|p`;|93IV3}GU?lh@e57YW*4#r|jW5bqcN)%k$R(VY9CkMn@c&nCw)YvAGKF-;P0K36V%o;2gp!GEkj0374kBNWRq{`V0j7jG-_^2FwE8-trg>t8&sbP`nj5MJy@H@vKQ~T%hdPdsyWh0k6eVz)^zmbMsNR_FQFb|BDcEWLmqm&7MJzne&Q${fBqdep+XyEt$qA2XA!fX%Sl%XAhoxB>tQ<Jgv4bNWFS;mZLMYr5%b_gdWsC~#|t8M*9x^e)1cR!8`j$*!<=n+X*h+?Drbv*)PYo4b9(`*~AQH2=21{iH^jPcy;(({i$h+XX9w`xGy8T<Cd=YWrH4_3%nyN-`mW}2Po=E?_!vy544MBA^^u(zuBd2!2U&}nX_4x@uK8Np_wvv`y@FTpK!2G|tTrnkbLiAI<XbVP*sJ<<Ld4*C3D8>G~H&FUd>Tw(c;s{^DCo$iG*%QqozBIY(-eysgJ!V*urybPw&3p|C*dpIG-49HG|<eVE7(+Rl^!Z$T1E1iE6F|kRKMu`l_Xj@29HdY-jxwwIq3>Tm-bZ>!3T2KFNo_<|u$<1eC7*C+$RySHs0H2ci+3NoGGR64OU41Bf-R(N=pv+s}jm#SJ4b=iTR|${g4oToPfw5v4D6z=c{1-IQJWK5?-*0tZzfMrQUjhqI1-OBAOGTa$8QH=>hDHAMU7Dav?9J^tTBx6iT!OOcp!483*;9%5Vd+BB2>;+Nh|61t&72XYk9g_PA|hXA_dVIO^2mQnS&huJq7M(<)ksr<KhG88pNp-pGs?K<^RzJ_V#5w66El*Q9gH`@FTCJ1%9Eee`H)+nuQ9#d)-}-nLZasyrXl~khc{kR(2Zg4dsNxLWR|C;4JE(Ph%lBng;R$|KOST~7Esb66lX=j8&j(%fL9iTL>yG<#WaDS9T<t84VHZZJ{R&gG7kBk7#}jA>>_!wnkQYTYk@=Dp@#?XBCS2QPA#i!2M7dYz!x;@2(jZAKk4hPXYfu_$%r!``5}jQa3+_!U;<!Sfzm+@`kzTuPP*|->7@<qs-<u55z#FOdQfZJC4N;n9Etb#4LG<_4xOJ-VBLFSPdy$Za?=eq{~>pUxg|Xp{<(z(`0Zd`_C~r452>UhbH(Zml4<b@A2zdu-OftWvBg8(C6@3y@#nedkV&Q-e~l~enM98rR3@Hkqy0Js^_$h+G$39ndYU-&j&z(r_=Js7vr5wwG#}$I0tv9Un>?A-aqolw%EIE7YO<LjKRy`MB{^R=@88>DZ-YaNY$9&j+OsVuFK?P6_!;DW-PP>2*UJ$BoLWbOnM*wysG{coY!HmL3sB(XfWINT`zpuErPWkB^NC+p)5>@dAV_#W%i1NICfK{J8|qiX69_HcIi_?x6gGwIrj;_V{i+-i`It1CXSJ7dPM<#<S2U=Epj@c^YcyA?%{bG_^MD__>y(jsGD%C(QR0-?$Ud2T6is7&B3gH6l|B#~J?|C%Ws=nIC-hLJh<fn8FA^gvu}TlVF~eW`3ANUD5BT;KQ?j5Jtwt=g@X#HHQf0B37)b`z8j%tkd$~kQ9%lE*KB_*d<T=v|$Q+ip0z~V{);u5v&y}}$r$rk87r#Jcf|FE}rN~J^JggwlVte%{DY91^RItofm2t!&_2v`v@?6SY6=Xg=jO{jRh-!2;K4CAFayo-a;Ii2*iC>D49|HWl%Z#=}YB_}szFR~MoFkLE*)E9oN)ScJJneROmMI>+Kcsx&P;Z!p*UE>gh*ACDtJxc;3&m(+hGM}&dUVDK^Vy3o%&9SRT`<(J^03a*TiKr;#X>*`;Ue#|S-RhZ-=#f0|Asr4g+qY@)xe*w*6)o&`ZGkApiD7=af+_#kg-9Zs-^0E;G;)|GudIm*@8v-72jv`tQQtrht<;1-lvlB-feUvu!JLYIG)01OG#sY4w+$9vZf-I+q9~ZPx#rMbD00)g@&G#f?~}0A{7c!QA}|90q3nxTwk#9To6<u%&5EYq_pH0!x1xtY|9i!BHTScjsH{aq$v0GfNAD#UwML``}hwl5_`=5j~9=Ofe6VcL3fKF9*}MG`Y$I+XqQo%^&|RPUIl8sWh-C*#^6#YRhb2|Hv8ApEw9Z+h^Zk3%gz();en~c)Q<}do+6AmxQBfmu3{3%p&vUAE{0!vk!Ux`M$5|x6rm)ker#`Y&5XciEXP*!L4TZVbeK`3`{&%@PFb|%lFwMJ_)E^ZRV&3CC=DH2M~v8;8-QzZupR{IKXjA!^!E_ae&KOaowVR7=WiefZ~Q5HylT^tB07IYT7gZ0fiFuQ_jVjtr+O2|cwkJU`uR6nZqQrrq>}4dY8Ga$?LBxNOaSEuY{OHmlJlK|tnRdJNw-1ge8StII!bgca)(`^QMp1bMbbCg#1`ItC@EB2!__HP5KRf=n6L@XJLADYTr+%nUZH7)AKU)~Yc+at%TxCq4IY8*Ie4~AZo@nT?Or&0SG(A=dNMB1?&}EhO9pSnrS@nS!ucE{RoL#4eiFxq3j#PMGK#MGc~Qe`-fZygB2W6HSD6$>@`s##{Hu#(gI_^I;<B+mTT8u^XuzPIo0uWUd*3yZ*rZ<Sb7Bb}<|&>2WMlmBDWtHT<L~HZS@H)>i#mW4Ri0L<ZG&~)QD{3I>Mf2MhFcSVo2Q64D5N|pUwegAvZLj<mG=E@!YuBA(}pP+ym7rg&|3~~0F=bfQ5PW{!}&1RNr+<ItH*)+Pb>Gzcdnx(_5RFSf-j5;U;PsO)xI&K&Iqotoh@T<sp1sGc#vosPbN_$c~hbh$5O9=nuitla6+%q0@1{NrBTJy+`6VM7GD7N@%x8m-w4zL?Tz}s7Jp>=k5ic){WIKx4%6Vsw*4yp$iGhglN0$JaOO{nDYLI$C(H>+s!~gsP--+Mlg;Bst{KYC!921YDf9JcnMRZ<K?F`NDzRww=w%tsUFzJd;h@C7j&CUwDXVyWR5QnI_k*pMP(w@Xq;OEaIl>U(rSJ&yv@2Jr-inbpkT?0DnkodTruM^?V#`RVVcEOaXXJ<=TAP3*P+3X97Zs>mZ{!UELJh@q<@#B&C!cvv=!+IB_92A~M@K0HK|EOjJ(;cpv?!c#){r>U^RVU?+Hcq*#Xg*8{Wk{zX?H&xlX=*#t)0OGtBogS4pkAveBWB}UjM_hQ6wV89pkoMY4dW_Z3<lnS^*)ZbXB~73rF+tv;5(Vf&Sw*4fREnHnDN{VT<_O6H1QkifsT?0|+hX11M`@&T8l5yMc>(y(^W@HNq3(VE=GP&D_0MFiK$!g)*&ldm`jOU{xf0XhpfPrX=)?ic4^1pJ%&m5gTb{4ckh>ju_SSrQXz1-p#y2k_5{}_15P`TN6On^K3Da)^<iYF_At$9tSC(1-69G#n&uAn1RE1ReNw@T~zxAGL`?p<kNO1x+!i8GR4bC_TTyPdw3M}EiDEi1PBRt4*mKnE<H6@hLtVdx3-Ql{zoR9z3|2M65;R-`|?`l>~Q=}(Q8y`Zg0}a2u^i0bkpWzp$P*Ex}K-NU&-M3!9?!-Mw#rZH1UuqKxy2Yisr)!sVQ<^ycj9<K{Rw)9Xt4KNEZI67H1~iD}V?s?UfqhF>Sz`Of)SBQiKzXD~mD6dO~y-socJ&-kV1MxZsUYyy?I<DW%z*HjY!8C18j0VC@mo%!-uS;O!cTVXgj7XkwgN4%)&J@;#vG5Ws)cdct!jRkA~EhOp;DupTLp+pelWh7a^CM*V%}iMKn6qPHa=O5TBOkMJ3A#7l2`%EoqwRrLHxRHQD&mD{G^T6;@@gwTrj7q8<cO@b3n%3H{gCRhtG|5Mw*@^>$TnkPV~WD)WxYnB2b7~J%0i(L)e2AQtE#D#@nxY(}NLav$iH|5GAcgZ2Y2Cl#FsFhW}oiPI7r|<u+qM?vd@Bx7QNk=nEhRP-Y"),format=L.FORMAT_RAW,filters=[{"id":L.FILTER_LZMA2}]))
diff --git a/records/track_10min_16mb/2026-04-04_MuonEqR_3LayerRecurrence_WD095_MLR022_AllInt6/train_seed0.log b/records/track_10min_16mb/2026-04-04_MuonEqR_3LayerRecurrence_WD095_MLR022_AllInt6/train_seed0.log
new file mode 100644
index 0000000000..0d7e8dd69f
--- /dev/null
+++ b/records/track_10min_16mb/2026-04-04_MuonEqR_3LayerRecurrence_WD095_MLR022_AllInt6/train_seed0.log
@@ -0,0 +1,221 @@
+
+*****************************************
+Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
+*****************************************
+Hyperparameters:
+  adam_eps: 1e-08
+  adam_wd: 0.02
+  beta1: 0.9
+  beta2: 0.95
+  compressor: brotli
+  data_dir: /home/dex/parameter-golf-with-cc/data
+  datasets_dir: /home/dex/parameter-golf-with-cc/data/datasets/fineweb10B_sp4096
+  disable_layer0_attn: False
+  distributed: True
+  ema_decay: 0.997
+  embed_lr: 0.6
+  embed_wd: 0.095
+  embedding_dim: 512
+  eval_seq_len: 2048
+  eval_stride: 64
+  gptq_calibration_batches: 64
+  gptq_enabled: True
+  gptq_reserve_seconds: 10.0
+  grad_accum_steps: 1
+  grad_clip_norm: 0.3
+  head_lr: 0.008
+  is_main_process: True
+  iterations: 20000
+  ln_scale: True
+  local_rank: 0
+  logfile: logs/9ad94044-4030-491d-a000-f621565297c2.txt
+  logit_softcap: 30.0
+  matrix_lr: 0.022
+  max_wallclock_seconds: 600.0
+  min_lr: 0.0
+  mixed_quant: True
+  mlp_mult: 4.0
+  model_dim: 512
+  model_path: final_model.pt
+  muon_backend_steps: 5
+  muon_beta2: 0.95
+  muon_momentum: 0.99
+  muon_momentum_warmup_start: 0.92
+  muon_momentum_warmup_steps: 1500
+  muon_wd: 0.095
+  muoneq_mode: r
+  n_int6_layers: 66
+  num_heads: 8
+  num_kv_heads: 4
+  num_layers: 11
+  parallel_residual: False
+  parallel_start_layer: 7
+  parallel_start_layer_is_physical: True
+  qk_gain_init: 4.0
+  quantized_model_path: final_model.int6.ptz
+  rank: 0
+  recur_layers_str: 3,4,5
+  recur_start_step: 3000
+  recur_warmup_steps: 20
+  repeat_untie_mlp: none
+  repeat_untie_mlp_layers: 
+  rope_base: 10000.0
+  rope_dims: 16
+  rope_train_seq_len: 2048
+  run_id: 9ad94044-4030-491d-a000-f621565297c2
+  scalar_lr: 0.02
+  seed: 0
+  skip_gates_enabled: True
+  sliding_window_enabled: True
+  tie_embeddings: True
+  tied_embed_init_std: 0.005
+  tied_embed_lr: 0.03
+  tokenizer_path: /home/dex/parameter-golf-with-cc/data/tokenizers/fineweb_4096_bpe.model
+  train_batch_tokens: 786432
+  train_files: /home/dex/parameter-golf-with-cc/data/datasets/fineweb10B_sp4096/fineweb_train_*.bin
+  train_log_every: 500
+  train_seq_len: 2048
+  val_batch_tokens: 524288
+  val_files: /home/dex/parameter-golf-with-cc/data/datasets/fineweb10B_sp4096/fineweb_val_*.bin
+  val_loss_every: 4000
+  ve_dim: 128
+  ve_enabled: True
+  ve_layers: 9,10
+  vocab_size: 4096
+  warmdown_frac: 0.667
+  warmup_steps: 20
+  world_size: 8
+  xsa_last_n: 11
+train_shards: 143
+val_tokens: 45514752
+model_params:34401371
+parallel_residual: active=0 start_layer=7 start_mode=physical params=0
+recurrence: layers=[3, 4, 5] start_step=3000 active=0
+repeat_untie_mlp: mode=none layers=[] params=0
+gptq:reserving 10s, effective=590000ms
+[rank1]:[W403 15:43:41.333694228 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+[rank3]:[W403 15:43:42.759532074 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+[rank4]:[W403 15:43:42.761293104 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+[rank5]:[W403 15:43:42.761524637 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+[rank0]:[W403 15:43:42.767210602 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+[rank6]:[W403 15:43:42.787436053 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+[rank7]:[W403 15:43:42.801718548 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+[rank2]:[W403 15:43:42.803494890 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+warmup_step: 1/20
+warmup_step: 2/20
+warmup_step: 3/20
+warmup_step: 4/20
+warmup_step: 5/20
+warmup_step: 6/20
+warmup_step: 10/20
+warmup_step: 20/20
+recurrence:prewarm active=1 virtual_layers:14
+recur_warmup_step: 1/20
+recur_warmup_step: 2/20
+recur_warmup_step: 3/20
+recur_warmup_step: 4/20
+recur_warmup_step: 5/20
+recur_warmup_step: 6/20
+recur_warmup_step: 10/20
+recur_warmup_step: 20/20
+0/20000 val_loss: 8.3145 val_bpb: 3.6139
+1/20000 train_loss: 8.3158 train_time: 0.0m tok/s: 8414190
+2/20000 train_loss: 12.3079 train_time: 0.0m tok/s: 8322806
+3/20000 train_loss: 10.7562 train_time: 0.0m tok/s: 8223119
+4/20000 train_loss: 9.0100 train_time: 0.0m tok/s: 8183649
+5/20000 train_loss: 7.8375 train_time: 0.0m tok/s: 8156259
+500/20000 train_loss: 2.9988 train_time: 0.8m tok/s: 7923791
+1000/20000 train_loss: 2.9482 train_time: 1.7m tok/s: 7921373
+1500/20000 train_loss: 2.9088 train_time: 2.5m tok/s: 7919597
+2000/20000 train_loss: 2.8387 train_time: 3.3m tok/s: 7917465
+2500/20000 train_loss: 2.7216 train_time: 4.1m tok/s: 7915663
+3000/20000 train_loss: 2.8276 train_time: 5.0m tok/s: 7914736
+recurrence:activated step:3000 layers:[3, 4, 5] virtual_layers:14
+3500/20000 train_loss: 2.6979 train_time: 6.0m tok/s: 7653592
+4000/20000 train_loss: 2.6219 train_time: 7.0m tok/s: 7468748
+4000/20000 val_loss: 2.6481 val_bpb: 1.1510
+4500/20000 train_loss: 2.5760 train_time: 8.1m tok/s: 7310121
+5000/20000 train_loss: 2.6197 train_time: 9.1m tok/s: 7207245
+5362/20000 val_loss: 2.5274 val_bpb: 1.0985
+stopping_early: wallclock_cap train_time: 590082ms step: 5362/20000
+peak memory allocated: 32484 MiB reserved: 32518 MiB
+ema:applying EMA weights
+pre-quantization post-ema val_loss:2.52510077 val_bpb:1.09752153 eval_time:2148ms
+Serialized model: 132405891 bytes
+Code size: 20700 bytes
+GPTQ:collecting Hessians from calibration data...
+GPTQ:collected 66 Hessians in 10.4s
+mixed_quant: sensitivity ranking -- 66 int6 (top), 0 int5 (bottom)
+  rank  0: int6 blocks.0.mlp.proj.weight sens=4291895808.0 numel=1048576
+  rank  1: int6 blocks.1.mlp.proj.weight sens=1131829504.0 numel=1048576
+  rank  2: int6 blocks.3.mlp.proj.weight sens=639102976.0 numel=1048576
+  rank  3: int6 blocks.2.mlp.proj.weight sens=533221920.0 numel=1048576
+  rank  4: int6 blocks.4.mlp.proj.weight sens=346102656.0 numel=1048576
+  rank  5: int6 blocks.5.mlp.proj.weight sens=297923904.0 numel=1048576
+  rank  6: int6 blocks.7.mlp.proj.weight sens=96664504.0 numel=1048576
+  rank  7: int6 blocks.6.mlp.proj.weight sens=95330528.0 numel=1048576
+  rank  8: int6 blocks.8.mlp.proj.weight sens=52402352.0 numel=1048576
+  rank  9: int6 blocks.0.attn.c_q.weight sens=50334144.0 numel=262144
+  rank 10: int6 blocks.0.attn.c_k.weight sens=50334144.0 numel=131072
+  rank 11: int6 blocks.0.attn.c_v.weight sens=50334144.0 numel=131072
+  rank 12: int6 blocks.0.mlp.fc.weight sens=50330312.0 numel=1048576
+  rank 13: int6 blocks.9.mlp.proj.weight sens=36778616.0 numel=1048576
+  rank 14: int6 blocks.0.attn.proj.weight sens=30242380.0 numel=262144
+  rank 15: int6 blocks.1.attn.c_q.weight sens=25165376.0 numel=262144
+  rank 16: int6 blocks.1.attn.c_k.weight sens=25165376.0 numel=131072
+  rank 17: int6 blocks.1.attn.c_v.weight sens=25165376.0 numel=131072
+  rank 18: int6 blocks.1.mlp.fc.weight sens=25165286.0 numel=1048576
+  rank 19: int6 blocks.3.attn.c_q.weight sens=25165124.0 numel=262144
+  rank 20: int6 blocks.3.attn.c_k.weight sens=25165124.0 numel=131072
+  rank 21: int6 blocks.3.attn.c_v.weight sens=25165124.0 numel=131072
+  rank 22: int6 blocks.3.mlp.fc.weight sens=25165122.0 numel=1048576
+  rank 23: int6 blocks.3.attn.proj.weight sens=23067072.0 numel=262144
+  rank 24: int6 blocks.4.attn.proj.weight sens=20999424.0 numel=262144
+  rank 25: int6 blocks.4.attn.c_q.weight sens=20133216.0 numel=262144
+  rank 26: int6 blocks.4.attn.c_k.weight sens=20133216.0 numel=131072
+  rank 27: int6 blocks.4.attn.c_v.weight sens=20133216.0 numel=131072
+  rank 28: int6 blocks.4.mlp.fc.weight sens=20133208.0 numel=1048576
+  rank 29: int6 blocks.5.attn.c_q.weight sens=16778130.0 numel=262144
+  rank 30: int6 blocks.5.attn.c_k.weight sens=16778130.0 numel=131072
+  rank 31: int6 blocks.5.attn.c_v.weight sens=16778130.0 numel=131072
+  rank 32: int6 blocks.5.mlp.fc.weight sens=16778130.0 numel=1048576
+  rank 33: int6 blocks.2.attn.c_q.weight sens=16776920.0 numel=262144
+  rank 34: int6 blocks.2.attn.c_k.weight sens=16776920.0 numel=131072
+  rank 35: int6 blocks.2.attn.c_v.weight sens=16776920.0 numel=131072
+  rank 36: int6 blocks.2.mlp.fc.weight sens=16776912.0 numel=1048576
+  rank 37: int6 blocks.10.mlp.proj.weight sens=15877998.0 numel=1048576
+  rank 38: int6 blocks.1.attn.proj.weight sens=15654982.0 numel=262144
+  rank 39: int6 blocks.2.attn.proj.weight sens=13337374.0 numel=262144
+  rank 40: int6 blocks.5.attn.proj.weight sens=11676767.0 numel=262144
+  rank 41: int6 blocks.6.attn.c_q.weight sens=7191526.0 numel=262144
+  rank 42: int6 blocks.6.attn.c_k.weight sens=7191526.0 numel=131072
+  rank 43: int6 blocks.6.attn.c_v.weight sens=7191526.0 numel=131072
+  rank 44: int6 blocks.6.mlp.fc.weight sens=7191522.0 numel=1048576
+  rank 45: int6 blocks.7.mlp.fc.weight sens=6291325.0 numel=1048576
+  rank 46: int6 blocks.7.attn.c_q.weight sens=6291324.5 numel=262144
+  rank 47: int6 blocks.7.attn.c_k.weight sens=6291324.5 numel=131072
+  rank 48: int6 blocks.7.attn.c_v.weight sens=6291324.5 numel=131072
+  rank 49: int6 blocks.8.mlp.fc.weight sens=5592424.5 numel=1048576
+  rank 50: int6 blocks.8.attn.c_q.weight sens=5592423.5 numel=262144
+  rank 51: int6 blocks.8.attn.c_k.weight sens=5592423.5 numel=131072
+  rank 52: int6 blocks.8.attn.c_v.weight sens=5592423.5 numel=131072
+  rank 53: int6 blocks.6.attn.proj.weight sens=5141315.5 numel=262144
+  rank 54: int6 blocks.9.attn.c_q.weight sens=5032689.5 numel=262144
+  rank 55: int6 blocks.9.attn.c_k.weight sens=5032689.5 numel=131072
+  rank 56: int6 blocks.9.attn.c_v.weight sens=5032689.5 numel=131072
+  rank 57: int6 blocks.9.mlp.fc.weight sens=5032684.5 numel=1048576
+  rank 58: int6 blocks.10.attn.c_q.weight sens=4575790.5 numel=262144
+  rank 59: int6 blocks.10.attn.c_k.weight sens=4575790.5 numel=131072
+  rank 60: int6 blocks.10.attn.c_v.weight sens=4575790.5 numel=131072
+  rank 61: int6 blocks.10.mlp.fc.weight sens=4575548.5 numel=1048576
+  rank 62: int6 blocks.7.attn.proj.weight sens=3795631.5 numel=262144
+  rank 63: int6 blocks.9.attn.proj.weight sens=2910124.0 numel=262144
+  rank 64: int6 blocks.10.attn.proj.weight sens=2810890.5 numel=262144
+  rank 65: int6 blocks.8.attn.proj.weight sens=2492911.0 numel=262144
+mixed_quant: most sensitive=blocks.0.mlp.proj.weight (4291895808.0), least sensitive=blocks.8.attn.proj.weight (2492911.0)
+GPTQ quantization: 66 layers with full GPTQ, 0 fallback to clip-search
+mixed_quant: 66 int6, 0 int5
+Serialized model mixed_int5_int6+brotli: 15935262 bytes
+Total submission size mixed_int5_int6+brotli: 15955962 bytes
+final_int6_roundtrip val_loss:2.54932435 val_bpb:1.10805018 eval_time:7204ms
+final_int6_sliding_window val_loss:2.50671417 val_bpb:1.08952989 eval_time:81110ms
diff --git a/records/track_10min_16mb/2026-04-04_MuonEqR_3LayerRecurrence_WD095_MLR022_AllInt6/train_seed42.log b/records/track_10min_16mb/2026-04-04_MuonEqR_3LayerRecurrence_WD095_MLR022_AllInt6/train_seed42.log
new file mode 100644
index 0000000000..93ec550230
--- /dev/null
+++ b/records/track_10min_16mb/2026-04-04_MuonEqR_3LayerRecurrence_WD095_MLR022_AllInt6/train_seed42.log
@@ -0,0 +1,221 @@
+
+*****************************************
+Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
+*****************************************
+Hyperparameters:
+  adam_eps: 1e-08
+  adam_wd: 0.02
+  beta1: 0.9
+  beta2: 0.95
+  compressor: brotli
+  data_dir: /home/dex/parameter-golf-with-cc/data
+  datasets_dir: /home/dex/parameter-golf-with-cc/data/datasets/fineweb10B_sp4096
+  disable_layer0_attn: False
+  distributed: True
+  ema_decay: 0.997
+  embed_lr: 0.6
+  embed_wd: 0.095
+  embedding_dim: 512
+  eval_seq_len: 2048
+  eval_stride: 64
+  gptq_calibration_batches: 64
+  gptq_enabled: True
+  gptq_reserve_seconds: 10.0
+  grad_accum_steps: 1
+  grad_clip_norm: 0.3
+  head_lr: 0.008
+  is_main_process: True
+  iterations: 20000
+  ln_scale: True
+  local_rank: 0
+  logfile: logs/04179401-7781-4aff-b0c2-a376fd05b843.txt
+  logit_softcap: 30.0
+  matrix_lr: 0.022
+  max_wallclock_seconds: 600.0
+  min_lr: 0.0
+  mixed_quant: True
+  mlp_mult: 4.0
+  model_dim: 512
+  model_path: final_model.pt
+  muon_backend_steps: 5
+  muon_beta2: 0.95
+  muon_momentum: 0.99
+  muon_momentum_warmup_start: 0.92
+  muon_momentum_warmup_steps: 1500
+  muon_wd: 0.095
+  muoneq_mode: r
+  n_int6_layers: 66
+  num_heads: 8
+  num_kv_heads: 4
+  num_layers: 11
+  parallel_residual: False
+  parallel_start_layer: 7
+  parallel_start_layer_is_physical: True
+  qk_gain_init: 4.0
+  quantized_model_path: final_model.int6.ptz
+  rank: 0
+  recur_layers_str: 3,4,5
+  recur_start_step: 3000
+  recur_warmup_steps: 20
+  repeat_untie_mlp: none
+  repeat_untie_mlp_layers: 
+  rope_base: 10000.0
+  rope_dims: 16
+  rope_train_seq_len: 2048
+  run_id: 04179401-7781-4aff-b0c2-a376fd05b843
+  scalar_lr: 0.02
+  seed: 42
+  skip_gates_enabled: True
+  sliding_window_enabled: True
+  tie_embeddings: True
+  tied_embed_init_std: 0.005
+  tied_embed_lr: 0.03
+  tokenizer_path: /home/dex/parameter-golf-with-cc/data/tokenizers/fineweb_4096_bpe.model
+  train_batch_tokens: 786432
+  train_files: /home/dex/parameter-golf-with-cc/data/datasets/fineweb10B_sp4096/fineweb_train_*.bin
+  train_log_every: 500
+  train_seq_len: 2048
+  val_batch_tokens: 524288
+  val_files: /home/dex/parameter-golf-with-cc/data/datasets/fineweb10B_sp4096/fineweb_val_*.bin
+  val_loss_every: 4000
+  ve_dim: 128
+  ve_enabled: True
+  ve_layers: 9,10
+  vocab_size: 4096
+  warmdown_frac: 0.667
+  warmup_steps: 20
+  world_size: 8
+  xsa_last_n: 11
+train_shards: 143
+val_tokens: 45514752
+model_params:34401371
+parallel_residual: active=0 start_layer=7 start_mode=physical params=0
+recurrence: layers=[3, 4, 5] start_step=3000 active=0
+repeat_untie_mlp: mode=none layers=[] params=0
+gptq:reserving 10s, effective=590000ms
+[rank0]:[W403 15:28:18.945408959 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+[rank7]:[W403 15:28:18.101870515 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+[rank1]:[W403 15:28:18.104905873 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+[rank6]:[W403 15:28:18.160996483 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+[rank4]:[W403 15:28:18.164076805 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+[rank5]:[W403 15:28:18.184024865 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+[rank3]:[W403 15:28:18.387737619 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+[rank2]:[W403 15:28:18.441843658 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+warmup_step: 1/20
+warmup_step: 2/20
+warmup_step: 3/20
+warmup_step: 4/20
+warmup_step: 5/20
+warmup_step: 6/20
+warmup_step: 10/20
+warmup_step: 20/20
+recurrence:prewarm active=1 virtual_layers:14
+recur_warmup_step: 1/20
+recur_warmup_step: 2/20
+recur_warmup_step: 3/20
+recur_warmup_step: 4/20
+recur_warmup_step: 5/20
+recur_warmup_step: 6/20
+recur_warmup_step: 10/20
+recur_warmup_step: 20/20
+0/20000 val_loss: 8.3184 val_bpb: 3.6155
+1/20000 train_loss: 8.3199 train_time: 0.0m tok/s: 8267695
+2/20000 train_loss: 12.3376 train_time: 0.0m tok/s: 8245994
+3/20000 train_loss: 10.7920 train_time: 0.0m tok/s: 8147372
+4/20000 train_loss: 9.0195 train_time: 0.0m tok/s: 8103277
+5/20000 train_loss: 7.8523 train_time: 0.0m tok/s: 8083136
+500/20000 train_loss: 3.0095 train_time: 0.8m tok/s: 7913617
+1000/20000 train_loss: 2.9458 train_time: 1.7m tok/s: 7913560
+1500/20000 train_loss: 2.9082 train_time: 2.5m tok/s: 7912616
+2000/20000 train_loss: 2.8413 train_time: 3.3m tok/s: 7910289
+2500/20000 train_loss: 2.7228 train_time: 4.1m tok/s: 7908058
+3000/20000 train_loss: 2.8287 train_time: 5.0m tok/s: 7906666
+recurrence:activated step:3000 layers:[3, 4, 5] virtual_layers:14
+3500/20000 train_loss: 2.6991 train_time: 6.0m tok/s: 7645544
+4000/20000 train_loss: 2.6258 train_time: 7.0m tok/s: 7461257
+4000/20000 val_loss: 2.6480 val_bpb: 1.1509
+4500/20000 train_loss: 2.5719 train_time: 8.1m tok/s: 7303595
+5000/20000 train_loss: 2.6152 train_time: 9.1m tok/s: 7200260
+5357/20000 val_loss: 2.5280 val_bpb: 1.0988
+stopping_early: wallclock_cap train_time: 590063ms step: 5357/20000
+peak memory allocated: 32484 MiB reserved: 32518 MiB
+ema:applying EMA weights
+pre-quantization post-ema val_loss:2.52552960 val_bpb:1.09770792 eval_time:2152ms
+Serialized model: 132405891 bytes
+Code size: 20700 bytes
+GPTQ:collecting Hessians from calibration data...
+GPTQ:collected 66 Hessians in 10.5s
+mixed_quant: sensitivity ranking -- 66 int6 (top), 0 int5 (bottom)
+  rank  0: int6 blocks.0.mlp.proj.weight sens=4551709184.0 numel=1048576
+  rank  1: int6 blocks.1.mlp.proj.weight sens=874587712.0 numel=1048576
+  rank  2: int6 blocks.3.mlp.proj.weight sens=713194176.0 numel=1048576
+  rank  3: int6 blocks.2.mlp.proj.weight sens=564713280.0 numel=1048576
+  rank  4: int6 blocks.4.mlp.proj.weight sens=312073376.0 numel=1048576
+  rank  5: int6 blocks.5.mlp.proj.weight sens=286648928.0 numel=1048576
+  rank  6: int6 blocks.6.mlp.proj.weight sens=140611536.0 numel=1048576
+  rank  7: int6 blocks.7.mlp.proj.weight sens=72432296.0 numel=1048576
+  rank  8: int6 blocks.8.mlp.proj.weight sens=55828284.0 numel=1048576
+  rank  9: int6 blocks.0.attn.c_q.weight sens=50332376.0 numel=262144
+  rank 10: int6 blocks.0.attn.c_k.weight sens=50332376.0 numel=131072
+  rank 11: int6 blocks.0.attn.c_v.weight sens=50332376.0 numel=131072
+  rank 12: int6 blocks.0.mlp.fc.weight sens=50330304.0 numel=1048576
+  rank 13: int6 blocks.0.attn.proj.weight sens=31715604.0 numel=262144
+  rank 14: int6 blocks.9.mlp.proj.weight sens=31607054.0 numel=1048576
+  rank 15: int6 blocks.1.attn.proj.weight sens=25385600.0 numel=262144
+  rank 16: int6 blocks.1.attn.c_q.weight sens=25165716.0 numel=262144
+  rank 17: int6 blocks.1.attn.c_k.weight sens=25165716.0 numel=131072
+  rank 18: int6 blocks.1.attn.c_v.weight sens=25165716.0 numel=131072
+  rank 19: int6 blocks.1.mlp.fc.weight sens=25165252.0 numel=1048576
+  rank 20: int6 blocks.3.attn.c_q.weight sens=25165114.0 numel=262144
+  rank 21: int6 blocks.3.attn.c_k.weight sens=25165114.0 numel=131072
+  rank 22: int6 blocks.3.attn.c_v.weight sens=25165114.0 numel=131072
+  rank 23: int6 blocks.3.mlp.fc.weight sens=25165110.0 numel=1048576
+  rank 24: int6 blocks.4.attn.proj.weight sens=21393270.0 numel=262144
+  rank 25: int6 blocks.4.attn.c_q.weight sens=20133226.0 numel=262144
+  rank 26: int6 blocks.4.attn.c_k.weight sens=20133226.0 numel=131072
+  rank 27: int6 blocks.4.attn.c_v.weight sens=20133226.0 numel=131072
+  rank 28: int6 blocks.4.mlp.fc.weight sens=20133220.0 numel=1048576
+  rank 29: int6 blocks.10.mlp.proj.weight sens=16802044.0 numel=1048576
+  rank 30: int6 blocks.5.attn.c_q.weight sens=16778134.0 numel=262144
+  rank 31: int6 blocks.5.attn.c_k.weight sens=16778134.0 numel=131072
+  rank 32: int6 blocks.5.attn.c_v.weight sens=16778134.0 numel=131072
+  rank 33: int6 blocks.5.mlp.fc.weight sens=16778126.0 numel=1048576
+  rank 34: int6 blocks.2.mlp.fc.weight sens=16776902.0 numel=1048576
+  rank 35: int6 blocks.2.attn.c_q.weight sens=16776880.0 numel=262144
+  rank 36: int6 blocks.2.attn.c_k.weight sens=16776880.0 numel=131072
+  rank 37: int6 blocks.2.attn.c_v.weight sens=16776880.0 numel=131072
+  rank 38: int6 blocks.3.attn.proj.weight sens=15994396.0 numel=262144
+  rank 39: int6 blocks.5.attn.proj.weight sens=14838362.0 numel=262144
+  rank 40: int6 blocks.2.attn.proj.weight sens=14622013.0 numel=262144
+  rank 41: int6 blocks.6.attn.c_q.weight sens=7191527.0 numel=262144
+  rank 42: int6 blocks.6.attn.c_k.weight sens=7191527.0 numel=131072
+  rank 43: int6 blocks.6.attn.c_v.weight sens=7191527.0 numel=131072
+  rank 44: int6 blocks.6.mlp.fc.weight sens=7191525.5 numel=1048576
+  rank 45: int6 blocks.7.attn.c_q.weight sens=6291326.0 numel=262144
+  rank 46: int6 blocks.7.attn.c_k.weight sens=6291326.0 numel=131072
+  rank 47: int6 blocks.7.attn.c_v.weight sens=6291326.0 numel=131072
+  rank 48: int6 blocks.7.mlp.fc.weight sens=6291322.0 numel=1048576
+  rank 49: int6 blocks.8.mlp.fc.weight sens=5592425.5 numel=1048576
+  rank 50: int6 blocks.8.attn.c_q.weight sens=5592421.5 numel=262144
+  rank 51: int6 blocks.8.attn.c_k.weight sens=5592421.5 numel=131072
+  rank 52: int6 blocks.8.attn.c_v.weight sens=5592421.5 numel=131072
+  rank 53: int6 blocks.7.attn.proj.weight sens=5144346.0 numel=262144
+  rank 54: int6 blocks.9.mlp.fc.weight sens=5032756.0 numel=1048576
+  rank 55: int6 blocks.9.attn.c_q.weight sens=5032699.5 numel=262144
+  rank 56: int6 blocks.9.attn.c_k.weight sens=5032699.5 numel=131072
+  rank 57: int6 blocks.9.attn.c_v.weight sens=5032699.5 numel=131072
+  rank 58: int6 blocks.6.attn.proj.weight sens=4768773.5 numel=262144
+  rank 59: int6 blocks.10.attn.c_q.weight sens=4575707.5 numel=262144
+  rank 60: int6 blocks.10.attn.c_k.weight sens=4575707.5 numel=131072
+  rank 61: int6 blocks.10.attn.c_v.weight sens=4575707.5 numel=131072
+  rank 62: int6 blocks.10.mlp.fc.weight sens=4575607.0 numel=1048576
+  rank 63: int6 blocks.9.attn.proj.weight sens=2782845.8 numel=262144
+  rank 64: int6 blocks.8.attn.proj.weight sens=2546981.0 numel=262144
+  rank 65: int6 blocks.10.attn.proj.weight sens=1546862.0 numel=262144
+mixed_quant: most sensitive=blocks.0.mlp.proj.weight (4551709184.0), least sensitive=blocks.10.attn.proj.weight (1546862.0)
+GPTQ quantization: 66 layers with full GPTQ, 0 fallback to clip-search
+mixed_quant: 66 int6, 0 int5
+Serialized model mixed_int5_int6+brotli: 15940329 bytes
+Total submission size mixed_int5_int6+brotli: 15961029 bytes
+final_int6_roundtrip val_loss:2.54976517 val_bpb:1.10824178 eval_time:7237ms
+final_int6_sliding_window val_loss:2.50733015 val_bpb:1.08979763 eval_time:81352ms
diff --git a/records/track_10min_16mb/2026-04-04_MuonEqR_3LayerRecurrence_WD095_MLR022_AllInt6/train_seed7.log b/records/track_10min_16mb/2026-04-04_MuonEqR_3LayerRecurrence_WD095_MLR022_AllInt6/train_seed7.log
new file mode 100644
index 0000000000..8cbfb6b788
--- /dev/null
+++ b/records/track_10min_16mb/2026-04-04_MuonEqR_3LayerRecurrence_WD095_MLR022_AllInt6/train_seed7.log
@@ -0,0 +1,221 @@
+
+*****************************************
+Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
+*****************************************
+Hyperparameters:
+  adam_eps: 1e-08
+  adam_wd: 0.02
+  beta1: 0.9
+  beta2: 0.95
+  compressor: brotli
+  data_dir: /home/dex/parameter-golf-with-cc/data
+  datasets_dir: /home/dex/parameter-golf-with-cc/data/datasets/fineweb10B_sp4096
+  disable_layer0_attn: False
+  distributed: True
+  ema_decay: 0.997
+  embed_lr: 0.6
+  embed_wd: 0.095
+  embedding_dim: 512
+  eval_seq_len: 2048
+  eval_stride: 64
+  gptq_calibration_batches: 64
+  gptq_enabled: True
+  gptq_reserve_seconds: 10.0
+  grad_accum_steps: 1
+  grad_clip_norm: 0.3
+  head_lr: 0.008
+  is_main_process: True
+  iterations: 20000
+  ln_scale: True
+  local_rank: 0
+  logfile: logs/f29511fb-a96f-47b6-aa45-7aecb7c6679d.txt
+  logit_softcap: 30.0
+  matrix_lr: 0.022
+  max_wallclock_seconds: 600.0
+  min_lr: 0.0
+  mixed_quant: True
+  mlp_mult: 4.0
+  model_dim: 512
+  model_path: final_model.pt
+  muon_backend_steps: 5
+  muon_beta2: 0.95
+  muon_momentum: 0.99
+  muon_momentum_warmup_start: 0.92
+  muon_momentum_warmup_steps: 1500
+  muon_wd: 0.095
+  muoneq_mode: r
+  n_int6_layers: 66
+  num_heads: 8
+  num_kv_heads: 4
+  num_layers: 11
+  parallel_residual: False
+  parallel_start_layer: 7
+  parallel_start_layer_is_physical: True
+  qk_gain_init: 4.0
+  quantized_model_path: final_model.int6.ptz
+  rank: 0
+  recur_layers_str: 3,4,5
+  recur_start_step: 3000
+  recur_warmup_steps: 20
+  repeat_untie_mlp: none
+  repeat_untie_mlp_layers: 
+  rope_base: 10000.0
+  rope_dims: 16
+  rope_train_seq_len: 2048
+  run_id: f29511fb-a96f-47b6-aa45-7aecb7c6679d
+  scalar_lr: 0.02
+  seed: 7
+  skip_gates_enabled: True
+  sliding_window_enabled: True
+  tie_embeddings: True
+  tied_embed_init_std: 0.005
+  tied_embed_lr: 0.03
+  tokenizer_path: /home/dex/parameter-golf-with-cc/data/tokenizers/fineweb_4096_bpe.model
+  train_batch_tokens: 786432
+  train_files: /home/dex/parameter-golf-with-cc/data/datasets/fineweb10B_sp4096/fineweb_train_*.bin
+  train_log_every: 500
+  train_seq_len: 2048
+  val_batch_tokens: 524288
+  val_files: /home/dex/parameter-golf-with-cc/data/datasets/fineweb10B_sp4096/fineweb_val_*.bin
+  val_loss_every: 4000
+  ve_dim: 128
+  ve_enabled: True
+  ve_layers: 9,10
+  vocab_size: 4096
+  warmdown_frac: 0.667
+  warmup_steps: 20
+  world_size: 8
+  xsa_last_n: 11
+train_shards: 143
+val_tokens: 45514752
+model_params:34401371
+parallel_residual: active=0 start_layer=7 start_mode=physical params=0
+recurrence: layers=[3, 4, 5] start_step=3000 active=0
+repeat_untie_mlp: mode=none layers=[] params=0
+gptq:reserving 10s, effective=590000ms
+[rank3]:[W403 21:41:12.827955687 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+[rank1]:[W403 21:41:12.038029758 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+[rank6]:[W403 21:41:12.129566098 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+[rank5]:[W403 21:41:12.157168827 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+[rank4]:[W403 21:41:12.158813571 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+[rank0]:[W403 21:41:12.162824119 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+[rank2]:[W403 21:41:12.164675727 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+[rank7]:[W403 21:41:12.187828449 reducer.cpp:1431] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
+warmup_step: 1/20
+warmup_step: 2/20
+warmup_step: 3/20
+warmup_step: 4/20
+warmup_step: 5/20
+warmup_step: 6/20
+warmup_step: 10/20
+warmup_step: 20/20
+recurrence:prewarm active=1 virtual_layers:14
+recur_warmup_step: 1/20
+recur_warmup_step: 2/20
+recur_warmup_step: 3/20
+recur_warmup_step: 4/20
+recur_warmup_step: 5/20
+recur_warmup_step: 6/20
+recur_warmup_step: 10/20
+recur_warmup_step: 20/20
+0/20000 val_loss: 8.3151 val_bpb: 3.6141
+1/20000 train_loss: 8.3170 train_time: 0.0m tok/s: 8411369
+2/20000 train_loss: 12.2873 train_time: 0.0m tok/s: 8309067
+3/20000 train_loss: 10.7759 train_time: 0.0m tok/s: 8222543
+4/20000 train_loss: 9.0604 train_time: 0.0m tok/s: 8169056
+5/20000 train_loss: 7.9080 train_time: 0.0m tok/s: 8139938
+500/20000 train_loss: 3.0074 train_time: 0.8m tok/s: 7929270
+1000/20000 train_loss: 2.9494 train_time: 1.7m tok/s: 7928133
+1500/20000 train_loss: 2.9115 train_time: 2.5m tok/s: 7925521
+2000/20000 train_loss: 2.8389 train_time: 3.3m tok/s: 7921666
+2500/20000 train_loss: 2.7241 train_time: 4.1m tok/s: 7918222
+3000/20000 train_loss: 2.8330 train_time: 5.0m tok/s: 7916461
+recurrence:activated step:3000 layers:[3, 4, 5] virtual_layers:14
+3500/20000 train_loss: 2.7022 train_time: 6.0m tok/s: 7645741
+4000/20000 train_loss: 2.6274 train_time: 7.0m tok/s: 7462101
+4000/20000 val_loss: 2.6500 val_bpb: 1.1518
+4500/20000 train_loss: 2.5714 train_time: 8.1m tok/s: 7308829
+5000/20000 train_loss: 2.6223 train_time: 9.1m tok/s: 7201478
+5358/20000 val_loss: 2.5296 val_bpb: 1.0995
+stopping_early: wallclock_cap train_time: 590068ms step: 5358/20000
+peak memory allocated: 32484 MiB reserved: 32518 MiB
+ema:applying EMA weights
+pre-quantization post-ema val_loss:2.52748499 val_bpb:1.09855782 eval_time:2151ms
+Serialized model: 132405891 bytes
+Code size: 20700 bytes
+GPTQ:collecting Hessians from calibration data...
+GPTQ:collected 66 Hessians in 10.5s
+mixed_quant: sensitivity ranking -- 66 int6 (top), 0 int5 (bottom)
+  rank  0: int6 blocks.0.mlp.proj.weight sens=4516961280.0 numel=1048576
+  rank  1: int6 blocks.1.mlp.proj.weight sens=1050532416.0 numel=1048576
+  rank  2: int6 blocks.2.mlp.proj.weight sens=582654528.0 numel=1048576
+  rank  3: int6 blocks.3.mlp.proj.weight sens=542776640.0 numel=1048576
+  rank  4: int6 blocks.5.mlp.proj.weight sens=329390240.0 numel=1048576
+  rank  5: int6 blocks.4.mlp.proj.weight sens=293990176.0 numel=1048576
+  rank  6: int6 blocks.7.mlp.proj.weight sens=94879072.0 numel=1048576
+  rank  7: int6 blocks.6.mlp.proj.weight sens=93510800.0 numel=1048576
+  rank  8: int6 blocks.0.mlp.fc.weight sens=50330276.0 numel=1048576
+  rank  9: int6 blocks.0.attn.c_q.weight sens=50329216.0 numel=262144
+  rank 10: int6 blocks.0.attn.c_k.weight sens=50329216.0 numel=131072
+  rank 11: int6 blocks.0.attn.c_v.weight sens=50329216.0 numel=131072
+  rank 12: int6 blocks.8.mlp.proj.weight sens=47104052.0 numel=1048576
+  rank 13: int6 blocks.9.mlp.proj.weight sens=33910444.0 numel=1048576
+  rank 14: int6 blocks.0.attn.proj.weight sens=31762820.0 numel=262144
+  rank 15: int6 blocks.1.mlp.fc.weight sens=25165424.0 numel=1048576
+  rank 16: int6 blocks.1.attn.c_q.weight sens=25165322.0 numel=262144
+  rank 17: int6 blocks.1.attn.c_k.weight sens=25165322.0 numel=131072
+  rank 18: int6 blocks.1.attn.c_v.weight sens=25165322.0 numel=131072
+  rank 19: int6 blocks.3.mlp.fc.weight sens=25165124.0 numel=1048576
+  rank 20: int6 blocks.3.attn.c_q.weight sens=25165116.0 numel=262144
+  rank 21: int6 blocks.3.attn.c_k.weight sens=25165116.0 numel=131072
+  rank 22: int6 blocks.3.attn.c_v.weight sens=25165116.0 numel=131072
+  rank 23: int6 blocks.3.attn.proj.weight sens=22432974.0 numel=262144
+  rank 24: int6 blocks.4.mlp.fc.weight sens=20133216.0 numel=1048576
+  rank 25: int6 blocks.4.attn.c_q.weight sens=20133202.0 numel=262144
+  rank 26: int6 blocks.4.attn.c_k.weight sens=20133202.0 numel=131072
+  rank 27: int6 blocks.4.attn.c_v.weight sens=20133202.0 numel=131072
+  rank 28: int6 blocks.4.attn.proj.weight sens=19457514.0 numel=262144
+  rank 29: int6 blocks.1.attn.proj.weight sens=19391306.0 numel=262144
+  rank 30: int6 blocks.10.mlp.proj.weight sens=18801116.0 numel=1048576
+  rank 31: int6 blocks.5.attn.c_q.weight sens=16778138.0 numel=262144
+  rank 32: int6 blocks.5.attn.c_k.weight sens=16778138.0 numel=131072
+  rank 33: int6 blocks.5.attn.c_v.weight sens=16778138.0 numel=131072
+  rank 34: int6 blocks.5.mlp.fc.weight sens=16778138.0 numel=1048576
+  rank 35: int6 blocks.2.mlp.fc.weight sens=16776881.0 numel=1048576
+  rank 36: int6 blocks.2.attn.c_q.weight sens=16776867.0 numel=262144
+  rank 37: int6 blocks.2.attn.c_k.weight sens=16776867.0 numel=131072
+  rank 38: int6 blocks.2.attn.c_v.weight sens=16776867.0 numel=131072
+  rank 39: int6 blocks.5.attn.proj.weight sens=12425408.0 numel=262144
+  rank 40: int6 blocks.2.attn.proj.weight sens=9971989.0 numel=262144
+  rank 41: int6 blocks.6.attn.c_q.weight sens=7191524.0 numel=262144
+  rank 42: int6 blocks.6.attn.c_k.weight sens=7191524.0 numel=131072
+  rank 43: int6 blocks.6.attn.c_v.weight sens=7191524.0 numel=131072
+  rank 44: int6 blocks.6.mlp.fc.weight sens=7191524.0 numel=1048576
+  rank 45: int6 blocks.7.attn.c_q.weight sens=6291326.5 numel=262144
+  rank 46: int6 blocks.7.attn.c_k.weight sens=6291326.5 numel=131072
+  rank 47: int6 blocks.7.attn.c_v.weight sens=6291326.5 numel=131072
+  rank 48: int6 blocks.7.mlp.fc.weight sens=6291325.0 numel=1048576
+  rank 49: int6 blocks.8.attn.c_q.weight sens=5592428.5 numel=262144
+  rank 50: int6 blocks.8.attn.c_k.weight sens=5592428.5 numel=131072
+  rank 51: int6 blocks.8.attn.c_v.weight sens=5592428.5 numel=131072
+  rank 52: int6 blocks.8.mlp.fc.weight sens=5592425.0 numel=1048576
+  rank 53: int6 blocks.9.mlp.fc.weight sens=5032706.0 numel=1048576
+  rank 54: int6 blocks.9.attn.c_q.weight sens=5032695.5 numel=262144
+  rank 55: int6 blocks.9.attn.c_k.weight sens=5032695.5 numel=131072
+  rank 56: int6 blocks.9.attn.c_v.weight sens=5032695.5 numel=131072
+  rank 57: int6 blocks.6.attn.proj.weight sens=4886438.5 numel=262144
+  rank 58: int6 blocks.10.attn.c_q.weight sens=4575392.5 numel=262144
+  rank 59: int6 blocks.10.attn.c_k.weight sens=4575392.5 numel=131072
+  rank 60: int6 blocks.10.attn.c_v.weight sens=4575392.5 numel=131072
+  rank 61: int6 blocks.10.mlp.fc.weight sens=4575392.0 numel=1048576
+  rank 62: int6 blocks.8.attn.proj.weight sens=3378261.0 numel=262144
+  rank 63: int6 blocks.7.attn.proj.weight sens=3284608.8 numel=262144
+  rank 64: int6 blocks.10.attn.proj.weight sens=3015524.2 numel=262144
+  rank 65: int6 blocks.9.attn.proj.weight sens=1573591.5 numel=262144
+mixed_quant: most sensitive=blocks.0.mlp.proj.weight (4516961280.0), least sensitive=blocks.9.attn.proj.weight (1573591.5)
+GPTQ quantization: 66 layers with full GPTQ, 0 fallback to clip-search
+mixed_quant: 66 int6, 0 int5
+Serialized model mixed_int5_int6+brotli: 15943318 bytes
+Total submission size mixed_int5_int6+brotli: 15964018 bytes
+final_int6_roundtrip val_loss:2.55127439 val_bpb:1.10889776 eval_time:7181ms
+final_int6_sliding_window val_loss:2.50901269 val_bpb:1.09052894 eval_time:81157ms