Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Record: SP8192 + Parallel Residuals + Score-First TTT — val_bpb 1.0822 (3-seed mean)

**val_bpb = 1.0822** (3-seed mean, std 0.0005) | **~15.99 MB** | 8xH100 SXM

## 3-Seed Results

| Seed | Sliding BPB | **TTT BPB** | Artifact |
|------|-------------|-------------|----------|
| 42 | 1.0857 | **1.0826** | 15,991,486 |
| 314 | 1.0854 | **1.0822** | 15,991,486 |
| 999 | 1.0849 | **1.0817** | 15,991,486 |
| **Mean** | | **1.0822** | |

Merged SOTA (PR #1019): **1.1147 BPB**. Delta: **-0.0325 BPB**.

## Novel Contribution: Parallel Residuals + Score-First TTT on SP8192

This submission adds **parallel residuals** (from layer 7) to the SP8192 + score-first TTT stack. Prior work had these separately:
- PR #1413 (@dexhunter): SP8192 + TTT, no parallel residuals → 1.0828
- PR #1412 (@Robby955): SP8192 + parallel residuals, no TTT → 1.0835

Combining both gives **1.0822** — better than either alone.

From layer 7, attention and MLP operate on separate residual lanes. A learned `lane_merge` scalar (init 0.5) blends the lanes after the final layer. This lets attention specialize on context mixing while MLP specializes on token transformations.

## Full Stack

SP8192, MLP 4x, depth recurrence (loop 4-5), parallel residuals (layer 7+), MuonEq-R, QK-Gain 5.0, SDClip, GPTQ embeddings, skip gates, score-first TTT (3 epochs, lr=0.005), brotli compression.

## Compliance (Track B — Score-First TTT)

- Score-first TTT: every token scored BEFORE weight update (PR #461 framework)
- No SLOT, no n-gram cache, no pre-quant TTT
- Model frozen after TTT; standard sliding-window eval
- All four conditions from Issue #1017 satisfied

## Reproduction

```bash
pip install brotli
MATCHED_FINEWEB_REPO_ID=kevclark/parameter-golf python3 data/cached_challenge_fineweb.py --variant sp8192 --skip-manifest
SEED=42 TTT_ENABLED=1 PARALLEL_START_LAYER=7 torchrun --standalone --nproc_per_node=8 train_gpt.py
```

## Credits

PR #1394 @clarkkev (SP8192 base), PR #1413 @dexhunter (score-first TTT on SP8192), PR #1412 @Robby955 (parallel residuals on SP8192), PR #1204 @msisovic (parallel residuals concept), PR #1260 @dexhunter (MuonEq-R + depth recurrence)
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"author":"aryanbhosale","github_id":"aryanbhosale","name":"SP8192 + Parallel Residuals + Score-First TTT","date":"2026-04-08","track":"10min_16mb","val_bpb":1.08217956,"val_bpb_std":0.00045881,"seeds":[42,314,999],"seed_results":{"42":{"val_bpb":1.08261339,"artifact_bytes":15991486},"314":{"val_bpb":1.08222600,"artifact_bytes":15991486},"999":{"val_bpb":1.08169930,"artifact_bytes":15991486}},"hardware":"8xH100 80GB SXM","pytorch_version":"2.9.1+cu128","technique_summary":"SP8192 + MLP 4x + Parallel Residuals (layer 7+) + Score-First TTT + Depth Recurrence + MuonEq-R + QK-Gain 5.0 + SDClip + GPTQ Embeddings + Brotli"}
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
import lzma as L,base64 as B
exec(L.decompress(B.b85decode("{Wp48S^xk9=GL@E0stWa8~^|S5YJf5;KRK%vt0l*n@VT6Qap3bt~@<3h>ok~)Km^%<bI~`7~^@P9dNt*OmJtouSV|m@^}~LJOY#q""coM@9BaGiY8ypPvdJq=NbK}E`t%**OWHq5Yg*w`LcO2`&Ki8P4h{e@}LyJM^e5fwE@n-bff1Ph*mUliW#yCOV*=I+v<n|QZ$y02z""iZN=i)3}Qx?Dtm=+{LAgGTC@~>c^ys%R{D_%yAk9-_tV7^coUOo3$w>`(`ci)t`2F7>r>Ltx>>S2CRw|7ov>Wn1e~_!RLQ=%V9g?"")G3yPsu%SBy!lj1PaC-x%dDmCDOZ^r^!)+WWz}ejKXTJ#^U6Ra!};QocHHXQC+4UM!QQ!-N5Xd|%~a(9)bTYIO+>B~8~@lqmri%^""qEkQUy074Rh6w7V_#^s9J-3BNA`G;qyR$LYcI?e+loZVWi~B$n=TKFp{%SeHYp{oNWh;U@Ahk8M2$OU%K8B$lb*dRQXd-GR_@*KA""ZdRdwSd#X_bO(lvJ3fp9Otblkh?o!zlDF02+sRjLV6IqG{ieQx44UY<d&*~wW)rT(c@2s{9hl6`y|y8rgmg1R#?1!{D}?K9zb-nK""3<LeSkh!LeUzlS8h1WHc?b{wRB8OkBB)(6yMkVv8B?0MGfkcPfl|~~$iXJcP(EqoeNOe4svf$SH_eUIMd3dtb|BSUb1vN`%al^P7""1leUBh%RAF5AXfk0tT4&iR1x?tv*Fqvv>(f20c)^AD5kE{7_@f9?Q-ePHMY$wCTcn5ij2k?>T>CFcZ<|5Bh`%hA!j2d4G(X-Bbwu""<(#drc<T8}FJXTYMR7vF8q3yez=cNGB8F1PdmPmj9VC!WV!^&%dKzWR96*{A7A)N7k}vCp0ML~vVXoZYuA*2FSP01TJ&$T%a|!qZ""Za~I^2Aiw;pbe|H(ZRgX7ZRW>k2`tR2eo<C*fPF=ua-Gjm$6ej#6$+#oQea}M^CL|C)@Vy#>$wi$p$UEHkQdFiFmlJR#zIG@3*sm""d<EnlP}3{vNpKXtGa{;0yoE9_J7iup*avf}g|D`*^A5(PhPU5<`XPgyoW{6*wx-$Tw0|x*z(c}SPuGvdSsAz?4>lqZ?s>Cn@<hX2""3WqFEHc>I!i44iGk>T1KUmKDUWEJXYFF3Mh*&Tbca$esa+z^`enxeV%UmK_#Ex_)>$lBJA(W<ErW)!h>j|4yV%J<~unPL@@@KfP=""NTcv-SVPiG3BDdu=*>C1izrS~RvqEe6Re7Xf)zp2fR3F%Ntl(>3N{Nxb8vzZkhK?{<eqK!tMa#OYP1o`Dy)oYNF+~|aGI-K__ESp""dyNRoBiL${AF%Y3@3k1X@$hLMe}7%sFB8m}td+;Q`z`$r_9*3{1$UQTe#oFI!vccSiPbRWIFLr-mwXM?#v8qe`<b9)Lt_?xb)W6c""rB_wD5gh+^W{R@lmx#6zLwV#@1_nD0R04T*I#{4@@S}Gp;1-ERl{@G-{F5lhlG6=tFCYU|eQx?l%p4d5Ezm2`arwE~zPsD}sRUO0""xbUP~oyVEPTNl%)&WbZml`mtOy;FDP$3ko_(|CeVf00qEdhx-gvUE+Yuqpc**EoY7l;!+1iCBbOU;F-Ercm^sJyDbEB}@mHjpX=#""ek6j6`6xQKlKMUUIO*x&B5UXBO8Ed;zG!R;;M4}IVr%XG1}&xg#w2IS@~uQ}MPf~3OwoOUoU99s?ta~{fd1EH<8~3wqPnjI2eYhy""Nr{VoPb{v`k>sdJ*S<LPyc2pY+cXLu|IhvkmT<qcMbAjFM}{!5rEupSXGwj?MND79r0I!9no+&y1R6O|Amr1sUY@%zWF7(bFbvdV""4^Ot<zxGp)4585+1F)9YhVA<`E^nB@GkBL_96rRw7Oi6Hh+<l&6YH~r*PLknA0-HMv8v&uj5;K%U;sh~f0%I*#>y2p=iolv46?tI""C=EiU8Pg_OBclNXueT8CfKN<R2!~1h$6@<*Wq!*jG4o|CkKZIra!RG!&$+_Uy}>>_gcMZT{nNc@rq}9ytq&fPOKt-x5!M>DdwApa""2Kd<nWl{_E-BsTn0`7urY($U51jArfK)gz&fsfuEbc$empxL-rG}yffZBT{ku)mZUD;?H@^0;@}BO6Gn*njm^$0dM@%FhD$!m`Va""c@mHwaE6n(;9&F-pK{)n(zs0WZ`AEuDpi@_1mak8n$Eya@AY>FhwJ_<Sqjj2W(Fw>Fz=+)LIFNqPpJ)|apq<T;|jjEHf5k1#=e!<""$yR-Km_w$1hm0FOVJ@u10dqjGw@MAs4Hkckj#wvhwC?`NK7#JL;fBeHdo<o@)!M#cFeG4Gjc;;8QfhEMr2nKLEn>qa*X9=9uufcW""!~NxGUTALr_QIU$|NWf7JrxPK;nPQMIZ0fa8-Q&{)|Z)^m6VtU4cDZqwuWA`^Xm7L&I`o~I&Qe`9|tCCP$3G>sq~W(pn7GotUAaw""-n!PV6QqT83$FAczS~pAV>lTn1({N<lEF~37bdB6EYA#PBWfz&l}_SNR$CY)(Z2MX7qrUmWffEXnLmZl7m}kCvItVPt75vG8iP}5""UHC0rzfP_yJW#h_6tO2*VG1Fg)GR8$4OW!Np6jFQ3SK}YB*dO?X{pDm*TFS?99OFfLg^2$#pSx`?Tk>(YY!s&4d8?FyQVbuHM1KZ""d_rC{;yM=1|2#JGx$F_)1Plk9NO5SN`>=>nD#*p+;^(+$30Ht4RcfLVU=xdqv-?Qa>QOfzUZi}Ezjp$IXw-CH&H$cy4_-ST^l@js""oxX)ZJS5N=yR&pEzETnWvVXD7a?63JK7}kY(JDo%F=ea!qu=<NWjzhWp6u|Iyo!i!5V&Gk?$gt19X+GH1|g0myG@1=Molh?j2h~K""!vY|iRucSj%z;9-F7ThZ3eKLH@re+X3`Dd+b+u5&;0lwOJyyhrS+MA}V7IFq(0I2x9VPuv+;}?Lx!=7@ZL*GBV?Y5nnYIqI-{!Lq""2QGRPnWJp%<U7Y%dTVwsVSX60rzh_K3*dQXqD+^#^}*W==+4f-*%%l|9(sBQ^FgCbQ-6xc5~&^*fFrGjc~u&IHl}fQkIvQ7iO`|U""T2K9cW8q`6{J%HIqO@rofXpT}C+=y9=X50!Vy1EDg~fW+L*sJfDAfMTGJ<K8N}C!<&vzT6oev#{LvA&3zISDqDL=I_J9G_6W2W$t""r4{5P)H&sKfM7}YWkUfo1<oJ3kfjx30P=QKHTf!1`|!)-4LT92FYgFz$cw3ZI2<s`wZ7AQ&gO&=Jb-x;<MFt)>H=vjd#qG1iu=;@""lPbEUaia^$+U-M>)81R3UMNdZ(9bd8_TNKDAl0a>*Hr(LVog~-0hf4X@4WmW6}3xW<4?+sl84oo%#ir@ABXOdR;&~98kf*0N8-Z9""uKrDR5S5qm6psaqp~CFo79+XaY?Y^xdSGSP0fgl-HycBoc|Hecal1~F;*j-x1&MN~b{2||Vto3iph+m!&1M{3-r&vt)FMe*mvJRH""3Fb^Fq2<ZBMV9aiPkl6*CS8sVi`avHR8Xe8XJ7b1qg?m*5_w;*G2K4Q3ob{Q)MAG6TX0>InUdS-V7Ss!o$veeLDle8CNv+_l{Avc""#AyptT%;{S4*iU*++=k*nj|MmFH9eM2p)4}$LyzKQDZ-tZS$Xyu4JfY6W_FJ{9itB^`e>V^3A9#aADf&-E|!@?y*JJyh$|64IN-;""^<!=4o3#(hBgI1+kdR5|_D+_H2p#d-ssBJE2oA+@!nI9}Q{ZG(d3`zxV-jvRW0Ux4-jez*FBk{!x3@IqLBI@sA%L}1teMrvf+rrG""QFShd<z<h?veh+K!mVvplPtQzl~IqM_Q+;Akjtjt2T;fcfPNy2?IoYu>N8t{JFLyeSW$7cfagBcr723liY(dGA<KuP^|1fZQjW@j""LohZ2VQ;@nabe@NCtM!^Wnupr)O0H0eKvhe)phiX@gH-zhb#l?gL;_Mu`5rCf_Vh%?MruOrEmF6InnESE+4^o;{Q9R_qiO13Px2q""cu6VVt8^$u93+NdfMRm=+*0T%-c4KM)?lI3;J>X6Fq;9`e*S<jkc};MCxV==#Q8)OVwz1B3sMZYdSjrzig+%io>(50egK<|&=#MX""LgW`M7Pf+%ady!#!iDuIKqoh8aVeQFExT9GstMwYm=6u-`i}mj$v4EVAM-%0eVln#-`g7-aLMo8fq1LIv#+nU21`9ux+4nEQp^sz""zS>q@J&qQ_LaUPR<~^UY53;p2bTFsTuEXu0a1w;HL1bQ{K0W8<-$@iLgS9AKQo|Hn_#ch1$imEH52KAm!|Vy_jc+32=jPSMc4F>Z""W>?<OV={iC{FJNua4j$Fsv0Tbr`zfourjLZUr01t`ea<=(lUL_NDedw@+_E`sr~m;A)XW|ai#H$pcY{HopQ#y82o<UZ;V15>|1CF""CBgpe>ROYcph+DQ4_;U5Moh{Du6NF0{}fM2BvCN>h#4Ed^H6w5e@EZuE5?W*#~q~u^;TC_@dZxw8C5^jN{5NcV;)&VP&q`=)I?`6""tQjrh5f4d7EN#blBog(kl8l0i7>;+oVPKnFo_Lzlq?069Rh$k5uB7~f9aAb+?RY3;i-??HUze8UAy@f~6%m)-;am56do1JTs89~L""mU_S0vS)ozD2M!cfeMk+fZ!SWK0=88kJsu_(aiPCKFnN$xSs3-c2T3>xFzLx1)N`~Hi{QFbCltjYGG+|<<<FO+AF)M@uIKzO_HO%""ct#}TJ1TKZg>G|u?_k91My7vpDYpo2(^yfE{K(fa6m1?+btlyMJjKRXgX&UFZthQ__VPw$zR3cgE3%R0WxBKS3?W#3kPY@rTC(uq""*~#Iik|hyPI9gS`E}a-BW`I|UiqM7`NJz8v==|t@UG}pK^T$g{5^}kLFo>}!^YRnV@@>jMj0(qh6pZ}8GFZbr5Txt9P_{@#HrDTZ""P6P1>E4K?(qmaND)K_jOanSa5{~UT4+94l?wePT}eO=^JGG8h^D=&-4+m&)S)8dq39F^hy8*CcY8(OJgd)nX*C7iR>?Q=ho&orQ3""$2np?DX#a+!V5pueiTv}wFEIn>Hn?N0pVq#8wmi3vIIYOH2@QT*HDgJwK?d6L&Nc1D?4tC9&m(0;}-Iu^GoAcK+C&&ThG6pey4wN""xs@!eCAf?r*lQXP=5@u}9fbo`=-l>ZazH2X2b1%gZ$H;~Ze}RZP?6HPV+e`<azRLxM+Q*hl;W+fOq8wf;?hW0z1xIZ_)J2*!a6C#""idEeM|8?Usql<4=ZcYYIAQf<)-J9!N4g^O5{=<ek$O+T^xSq!%pu7|}adYZWiJRwp;xNcG@OaW$`i3q&#?)#%^z}ozYw<PI6lOB}""Ig<j0y|at6Mc8g<2#6*%C+b>WB~Uo<-Ag*LahsLRGm(J`#Ehw>Wak*1h*T%?AvVA0Y3Yvz9MPOkck%UFTVMANm|3Y=J$H!tqwA{O""?aQhwm0nZG$cfHZi5nrcs*!ef5=?`58RNklyFo3w#tCf62C1*B>6E?QSd=ZATnc6+PDjs2T`nv}o9%GM4{1svstns*;EDk#4A|Ej""2a_>L=kvIbjR{-I6Rs6Z`qFe>-NcbJTC+0K0FYb<u3Tz(Xw`jMAkUInW*Du64B{?^C8Rl}Ondy*3kI@XW?i(6k11x~mF0mvW%%Gh""dQ<Mr9J8NeN~`^bDrR!F-f(gC9#uzr0Spr330mOe>5`^A_N3_nhwhFZSwTA%h&-MJ5w)Lo0%AX>o*KqmoT*ky9D$~hZ}EhUW(x$%""QNx%tV`}6>7qL}>y(mOUce>Kj?|3LOVqNAGLMz7t1KQiXgtuk$aSGxg8?gwdPp|Te2pE<*)+9baIgJAKE7_Lz;AlMcxRAu3MQh%{""E{w#E!1T&w+OrKv(!m7RvciFQO`TpgGq~#Gnh(DA$%$#BfPuabpxx;Z`p1NB!+NT8oF2SlN`a?U8GZD2S-Ev)b#Q8>DV9J@E+Q`)"">(Y7RF>WZ9807R_lc8<7Y&P8+w)krHdS9%7f${gDes~VG1H<69omEREvMY#Grp1j;^wa4eS}W+@?wcJ&T{z{-0#-JilR~g>O+n^Y""$c?S3^e<`Lo82(YD^_PoSvZF#^ZmLijD@~;0TX>ouHGq(&I}t@o-x?p{~e_44tV5Df|0H8hf?WDW`F|sA5Y186ctRGNi77;&&^O%""6?i7;Rh>iGZ=c`;@*Kr`deN6OoymRmH=+bS$sa0Lc?iifo}sZGGIS2hswcn&f9R5xwZ<$sLp3%k$e9Lu;$ek*LzJ_g%dy?+a}bt~""Dl9IFILeGwCyW81AchHv+#LFd0$DO^5y_24E~r!8O2&)<CL@JUC7%d?ad5PovVSBKz`<S87%B^cJW5mqb9cXj+%LU2|1Oo>9~b46""j~t+@&zaVfsn!(Ueb4x=?+d>$9li4h^CW>P9QkEmUz_m~e=x_6)U5wPoXtsGvHvGQlHSkmt{tt7QzgB_L~W|NoJweR^>fo{`7$rx""-+-I%BAtV0fFIGJcTaR#wlW@ZGPwtYMlcry7xY)WiBJx226e|`TK8M%v-4w$I+PK8NQy(V5KnT>jjK5;z<<LNb6Oiwfn4J!rB>aa""7oaiN*^%uLKhly#(lL3=jg6~6@^aLm-a>T!EbnW1pCk6rVRn}n#vC7`r$fR=*VJz9at$x>nZr}INf6#zz-i7s23!))bfB(%e9%nt""j~Mx*EYh<rE_qeJ=3rLbfM-sgg0|AOG*4P<9Ul_?wG@Qzt>ED3jO-P%b=A-gS!EmVmY8$lJs6w)SU4O$UNwI!*jamf2p_!AxRU=H"";>zGuWolR}y);3lr5u}`70#C#zqlLnp!WHedCB$w?}*_M@aC+K2~Q%~3YNc9BWf_)vfIIKBu-dMNnA*nMGQDrS!{(g%-Z|P>qD{&""`JeGgmvA!)-O(liEb__uOCH_85q0vlNNb8;7t2*~hjQAoM5q9k_BA!S9Mvm%@?2IqH*(7Elo~HhO8#%NmjWj08N*;DwI1{*K@@pd"">C<8c#H~C%Vp<o6(Tid`_b;BS_{(b+B!?emIjVtXVkZ8ANwRrujUaeJ`n6-icD!>hDpci^0Tgm1mBScEdHpVc8-RQ~IVMZ@m4=3L""d25vUaGaoXI=qqbTI$X&Xu9d{?Fh7SygpwJv&!hQREo2f;+k*mRCK-7ABRQq`K}r}o8IF^RbZ<kbl@T1H;@zrkp-ES*75EOe}O-E""0AND_GM_VyPj8elz6{7gg)+L5Jj!6OghASKBpZ4flgWalz;$Fi$*-x;jdnqd4}c%~U_#o>a_SX|#hL%BV^hwn0LF7PKx;kB-N83G""<ILN*SBU{FNxlVl;#yD)rF>{nHiw{uS1IS`ogjXUL-Az_r@(<9`fZ{Qwt~}N-OUr+*1efP=PrFC9Y4IjMqK=_EaRM&&ipO)qF)dv"")3ZKuSHIt9AAC((ilqT*Pfk@|W6#Q4P$^rvN}W;R|NfsossrySuL2Qsbt}CF(lQ-c(Y7K>&B3AF*jll6s3tQ_p%VJFmJ?Xr!aHWU""BUR~bWEt4&2@rnuRdAhh7-$AOq+HRse^V*j`DVhOh5OZazY3hj)w|DZ<1>!Dg7LmOqkWG<4O)7jzmpmhtq^VWB7rj<pwol{Y;&ac""4<~qGoR&x`&5X1g#=P@0>i!(wO2u?0)Jha5d0?0*42;~FZNwE+gQ_xteVU^OEsRS7n22#ruhu7D*8UEsw}DMk2Gic`E)84%U%!VI""azw3DHI^c0ZY9WKd2{7F4EhEE54{)sTGVC$A!u$Q>PXD&vQ>=gdLP@$rH^xi(46Y~l0B}(BkZiwgJG<kaY)j}Eue~fRbdPw%hspw""i>J4Vb0Yo&<>m4aODx}D@3VQSkw>*HiSkUgK`e&~w5oE!m#k$R=?zV?Hipk2N9FsY@mc~q%Zh@-9feaq()2ib6R7wOO%7MnFr+(D""g$dYSKt_^?f!=B}V?SIYkFvjrLrm{<4sODWZmCE|iaV~ryf>bHR#Jsd8T)nlEu@!ZxNkN@-Vt`{wERYNC<D2&&0kBzi=YSNg#jW&""q1i{>+u#QfVgRb!ZDVnsARk@bSH1pdP&QD1iRNa~ko3g@a-lV`b_JkUae;Ktg?dZ!l{wA-@4XcL7?pz?HtS!7TW9reOq4gCJ_~f7""TbX6P{adpsCMp}3BJ;9DpG5tr4;MG`fpwM(20<ZXem(j}9~3;{^~o%ckZQ46O+T@?Sdqht_WhZ>)hWvWB~_|`pv%zR_uA=xInLzf""Y?6J7#l=bA0TLb`jUEmTmm)su;+D8~m#FQQfuf5}{cK_w5iVM5!08ohy{m<vxcJCA!@gS&)c{nL+_doX@FvwP;hBdg3NtuWZ!}8>""*QQ@ehEhPYw;H<_+a^7ZiuHy7ga&%*Z2mLE5l7<ja)FMY;CbqS#`rp{l7eqY1C@*jhL3oqPI5c&+Ybu8M=l{>+lB~J-Jm@PZdy3&""7W=^RV;nd4z~DBzKqDL@X_8T^14M>f<20l`OHR#oE93|bj4R@-VGf#p!$>m}W$IjRi;EDfmcxQu(}y<}126{en1j~WwX?^kJZMyk""5HL0oy=E#nL&8nlt3w{Q4B0M=&3#k>Bel{gh~&_M^INv+X;!qmRkjQ$UPmgGx*Js96iZC-s|^;>v9%jU%?L%>`0Zs|&1yP#*#~sI""Fol4TYZI-!c)B!%`6R1&+!3vXy`*^pY4&(}9)dc&_|Znd&nq>CRW~VDvU5Y)0%T#N1gkFXKq(Tuo@cyKt$+w|{Dkj9a)@et`}~Tb""+mU}vt&pcTr)DDBq3BHtnznF_zZ!W!{AlF3@k84;+&;4sdMMC|!BdnNIb&^W=|a%5CPo#peY>e**x!J!Wh>nClChz?Fh^Tvm>whD""d*hH`v;!91!WF*^r!2y6E--V`f7q^TkvP-s<z&qaoijnqDR30P?WAg<ry#<$39j`nDUu-zO4q}*CvrZ$WLLK(5akOE<;pLG@5c{@""Pa|hp16QJ<8p1((CVpp2Dinf}ummlMKR63OHv4r!1P}tCX4&IQkfhR=`4y0sPXl<Op0CTlMrh4)F|W5BnDI5w%n?#(GxL~#c-n$=""i)}DXsLyv$sxjNTDQ%N7rE>9;*0-^!Hay=83}$8RDEHv;K+ixLgPBrJdXifpbUJbK!KWT<zB-w;-v^UGD1E%V=AO>xXe1t6${T*H""zZ*j-#7W$aPIG=WO+*6ojuE@{4k?jn!P$kjYK3yAUt=^~>ga`aLqd<$yukJ7pO-lP?7XN1K8hWl>!v@z8H!aD{Rvl*+u1vco;vGe""nr9K`?B|Ge*bQJ8tXaN-?P3p+&P^^c4d05I-na8b$)t~R7Nr@$0nP|Ld(QlJ(GK6E#B`KaG@?>%2xq^P%WPUWdxYgseTo>yg!OpL""MK_7w$Bt<HgtR8%AbTKW5}EuW-(Ev+HsiVLeC#UJPJ$19gf8gYM4+hFY0=OOMOT5H8-rW1-QLe0tsoc&J17{I)X|U4T#>1e&CuhL""RV9MCpKgc3r=JLqPW=ak{P!P=KFHHp;l>OsAUq@$b7~*Sy?#IQ`maEizHWxn;`FPOSM-J1^nzuu>|1#<xysrZgWDJ=9>w-Gr(aQp""$}z@{?FGHA)tBJKpi?Y69d9h1f-dw8^ZMI~th)6zg8Duyma=Q%+#?jS%9{yJBV6BcT@yak6VPg-3+os;s-3JCWqL*x^gNSFX>BG7""t+j*EZ;=!Ar|6w(O%3k>g0~kWlCR_k^`27oXUM0FEgQjrX=KEiXfih$$`RK4JWpuc!D3Y*(l%P(N)F={sUP--mJcRD#)nbl@Xl|2""ocVVCYg#b_Sg**ETL+KH)ar}&X7GO%!GKo)oLgbD*;|q_FiO1=$9w>9t8hibk8YsZd4~@sNbT03Uj4^Pmb+>_Q>a-CBX2T{ek6i)""5ciK?t^K1PuiC1nRP+%olX5!%F$?Vu4Bn~>9v8)u_~)rGB?NQGfuh6YBG`V;oNSfi@e-*EhtEpz#xP%7J$A7?0oUPF<7KQ4#KLUm""{=b$K$*kcXQ@6a56%|yXm6EGg(Gc{N3z^?JsEe!f<HleWX7Al0hphnfAHvjlK7)ej?13VI(G!hk6r=)Nmra`#BRQZ2$1ZU%>GHMs""{7R^Sf&3AMgOH`5d{ESGCR*38MTRy<NxXjqGnMjj{R7CdYXQRYaqWj|5J5^trAo9gxk}hK9hO1=S|M|I`D&L-OUQWHPEi*rJ_*sZ"";G|<S(4i$27aqFyAxG^Kvl0j#H2$u_7yOz(HTaqJq-TM`r<8vk(t3`uZejS1J4FTsEqpWl#8x*>NC~+{3j6=DJsz%x+v1_XH)MJI""T+6xy4HpHvq3!;s_@DAHLPYk{*Q1k{nP};kYwS_h+%3hPZ;4!BiA!xP)nbPDmX+y5TrktVI%9*OrXRo5zCrP-B)rLVZlAGD)S7H7""Yu3h|G!H_$zwRQ1Vz&WaP?ylB!svjMi@R-u8h>*$9}Lh^P;6X7>NNn>-i+>NK-n7XP~oiE;DUN!fmUx&z8-KtvxGak=1}PQ&H!~x""zC`FLhyDnBtk5G(J}I){QcA99T&Qz%aF&QrtE(rzEGa@b{K<}C`u1H@!_fd~0S;~+=Frrot!Jml7w4)Fo>)>88vZR&H@chj@d<6M""r5WMXRRV#3p%=_1C;cU_?FSB^&zT@@CLD+|BcR8FrQcCFy;#`*wCU`|wng!g=`&FV2RExFYf6AQa=vqYLvw$KScxq*7H9G}5EGsG""!ea7&pS<@%PC4}!gl|{M_b$t87hd2O#~!6;Af@gyrS1UfjVbO!!`fE<W^~T-fN4+(Q4<4^hVe+alim=8MIxm|-35co8^!pwy*UQC""5+Ix>DPVNU#l2nz+0TEmH{&-hNFT6+p(EW-gPcs=-41y3+(m3u^q{0fk#!mrS!5;J)i!QHCW(ecms#SxWT5Qj7;AL_|KFI)jy{Vm""Vm(sAW$0FQe1qk3=jkgfE-35O?SH!&vq<(61Ui|h@;H8<exET5uv+GmW5*4&+&)PbIBtb-3+<!i(wm@!I|5;ZA(g+2zY5G==BRi+""bT;n`9poDgvcyISSE}bCwG+9-GzJzi2QPmVn|~DHx<5$9lTq#`scWOzR9wWfn}Sh^PsfxVcX+IG?Agh7S~CCt@=Nky)LKn;o+|Lj""cqTP+(v~VUT%zTG<SRm<5xxE#=EMLhY5KD%3BTJV543-DHiI;~Wc2QHIa6-H4S<DmNWyM_ZXBpGTia!2DF`JhL@<TLQqSmp4C%%V""bNg_RZMy*GNcq>W2&tLUxh;{JHxmCto+|+xrSX<@VWiRvPHy%U)tO<AqK<V-ZTxUk(mazfYgqyTx`NXkFsZLlZCe|A6X*-U4}mB=""E02-s{)}_5yyZdc!v(LG=Tg}b<~V?tFxM#rO-<)&QH$MwI>WE)+^(kIBT4m1#mw5OX^MkZrd$H0*#GnT*URL4Xe4XNBw1F@8ElB1""hxD?q&2+L$X5d*g(3TK4u<(?kN1>8$GOj3K|GaGI#HhYwF0{CPagqADf_gDYJ_3G&Vl01$^jYOG%=m7HXIm`vPTca51^jd9RJNfz"">5nv<w!}I--mrwGx$z!3W6{LVbE-A#n6V4wS#i?fP20;B4K1d7_G1N|_$EM2WvuHG6esVb2QKX!dvV4?DA0#16W|xSVO9sPwWeQ_""9}xV^IS!7-Dv1l&znnYrXf_5sCy!tk=D~-44e>RT*1uw(;Sbfq8!{&_N3o1xP$f7oFzA6#-~V0$kn>6gAPx?#!Ys0Y<Z5x{X{jDc""diV74B~VGyOp5z)i^vhH8sv6Z!E#`(hiWTep*HAmtoV)rZOn1Kkvh8dk73?}o?<g&de-;0?e2-U#IvtHsr{NsuTq}KnQ-B{S|YEM""+7{8re!{3uDJq$cY?`#3(MlZe-rNknP3T&=JN1spK>0+(D5-P4GWmk+6>$A7Mw4shf0Lk4jRvTK@q1W-4jSeyzf{Ukgki$-<&s}4""Ft2OwE+_IePj>)mxq$pzOg{6;fk7}Dma&HwtwBJoW#{5cgC2%(LEg8ogm;y3$zWZ)gL6bwlobMiURrb8qmi|+*~raooeJEcicDoa""n>V>6A%4+x)=g0*e}%Xg{wPhDsv+cPdTtoW=G3jg1|9Q)9){Z(JwSaCj3uUEt-(~RPKwOnE!E*?=x<$UEz*vAGwf|<s5qvA`Nfpb""qEQ`kZ<!H@rN;qPw}%KrOXdIW=V1Vu^T)IDDX(SlR3gxgk-US^AF)$FNYKo)<n=_XlWGQBc-?^2Z4s3j^lLPFw^H##9tFrQqUL4d""<irGb_vxhT{v8Nf9$}e=5R(eemrS#aEd`4SZssg9VA9kAAC}*ING3RY1^*4n0Ul8Q_n$%SbIVY(qr*paag|$n=}FXJmwmfvFXZn*""Z$uC)p63cPpqqMo%wC}RA*c`O*MOpvakA)Obnx&5OGK=bCw$5QO^%bg9JpH7u+fd1BWN&MOqm^ZL9O1Ipx(W#+eME0cz7m1!o=mV""?g6!=BJ23H-KacT9IBq)9_9-8e61qM2u;a<yxtNMsHGn!w~_bt%Ans!N&67G<JaKh8enr^jYg+&SZrv2ga7noEZOt{EQ+5gpJnsB""{U89-sx#p!9)AFPFT_i=i2dj%QVJ{`gC;f(KYsp?ygR7)Bv5s0*ve-ze)G0}FPRkI%xLjnTM{mBkW@3eov6;Q@yFq$DTa~Pw~;d}""Gl*1<1MJ$o&DfO<f(K;*5GKs$hhZ^D{wPUk+zU1my;6PN=Nx;s*>G9hY}?qKSrkp)eT&>ik^=<Ghx6)o@8fgI0s#vCBn<zUS#S|T"";8E1ZT!rR8tm5|1bW%#X#A>6E6Y+<FfO?txuY#;vD!nL*y>P-Qdr2?wY3_05zIKezCs|)N-H$eXhOe_3tPsox+Q^TX8;287>byb%""Jz<m?M@z0!Ed1t|8a4-Vf%-mL{~6M=1pP_MTg|(U;UrOE&VDrav{SkD-n-LE(jX^TD<&LuxcZU|b=lK;M(5CdCi0yS^f}mQxH#|C""JriNp20Nm6LL=k7Zcfl!mc7^+Q~nsYD3i24LorY9)sDZ`Q>&p1C5+Ah?1S8hD4dQ{_`mVUH7yB2_igMZeEv{lvZ!C}KSlIq(Zc#*""E5lL0tiPv!0+a|ar9WS%NpN41hp-QeZ#$9TCVGB=k);6vIkS^HuZl{Q0x4gWmV<mb**#Iq>KEQ1jmLy;NbT=b;z(M~Rxbgire14U""jhrctxXA2Q@W8r=H^>~<=sMO;bhdIwqKwU*UD1iN=bv(tpM1y3?CeugP7Wg7VNKCSD<n{062fya*%=5-f4|5YQtSa7hJ0}iOhzf3""RaE7@$F&7DKc8O`b}y7;Y(}xRBv-xRLT6^8*TCTYKh{1O<PFOjNFw&5Sv@9L)*DB`=({ZbeLR$C6%LTrTr{=E;)bDCZEbY4@TQ(0"";i<vLkj|O6(d$>_o5)g!)t;`Aa9t^CU28xG_Drk9c5KfXTX=omM+q3~rANUF&)>J}Q#bYkc?rndA~8Is{g013p&x1V%U!V2AsVM-""YQYdRs7Kl<v4+H+KLbssR?ZKa5k(cXKq!|F|KmZcJBG+;nN0=U1<7xa7OPW#n~qNiKY6WbdR34v2XBs`&&5XFZo<>GNlz39BU}+V""A4?)vz(_B<N?*Qt<oTmd>T<|~pZdrgt(n7JZCBzYSyY%CJl39eb{$*x9%q+6Wwc}beCA9_=Wr>@d#@F_z)`d(-WboMpeq7n3Yu;t""reSL!{HM&Kh%*tW7i9%GkttE<wBCNb4uGjR$(PJ4)CcCNdXcvl5k@T62a>bBA%f`EWduc@<~d0)8%&B(8iCcRn0`S;7m+#S(~dz-""%S?veVTAvnqbXp&<T#4p8+Ed#*(BHkFcbV59-ztmz2!>c0FI%H0kIW4_JF`R1LV&4!kvzM@okxrRS(H1HS8Bd?;Y5<qi#6>Yg!vk""$J#t}`!J3rlO}RZsgqnxn@GkbBDvjfi&E(&yFZ(%(oTXCeTzfQw7+0Fbl7@0hBL+z2Omrm4rKhZ&+reE;0wDlTaL;3bQ6(CfA)B1""w1YK?Ts^Hyf^_GXe6vyR0XFh!+BXa}bk<2Y=G+L9Oi4&IE5ygmJ-8b5cYy!h%)mUVsoSOHnkG5?NYJn0r;9QLO7Mm5zGVB6ODvlG""{NvlQ)6g3J7&x+5jAspjy>CM&Qw*UYTqD)T%tq&Ko$$$l8h%tj--RZ<-z=mKa$pdM!4YSU7^G#>{Y;~YjdK2hDM$w1xkOn*XCwZ+""tm&jWAz0E<$kF&I(&qSX{i?YWY5DaPl18Z1z@MCpu{l05Nu%G|1e5Mf90uELQ7W^bUop5W;2Bv9W!)A?5-B_IGcrLE64LqgiFvtC""g}2LljhN8+ZC(It(+|p_@D~w>4dw&lV(fbI^f-FjOlsH>%*?zaRfhNRxUdlT5JkqFK7MMpT5I=|xWQNJv%)F32YZQU3YW@tg-!1U""##okRJ{s!I&<{!(w?8dy`NpN}rv??I>wL<z_vjvaM#&L7Ng_o@u(Kq8g315cE|Qvo^!0AG&2~%E2vN#W;v`N?Ib-=T^EG8uk02|c""`TZ3Ae@89EUz`<raIh-CTp#bdD*6beJG*WIVNjJtD6||2j3P}H$2{sS?Y*~@ia5eT2i)J(7ab&?KPM>!8_OPynvVa(nDr&&^SbV;""$FO_FpUJ$s>aTzTm44uv+6=vtCAbe%HBhhn8aXAGB={zY1zX6l{^mzw;BGsp0pC9A)A{gOZZ}S|jU13BWWet%U!uSIMfRCgI|W;}""W`!k{{C&T%jbZun5!}+PcdhZj0s;9r^t=a9u@E4))Bn>#;bVBx7EprlV+1{K;GI?@@!`;bF3_yk3&o*ZINL&m&zAs5@&F<99Z+Xg""eLU(%En)?*cUXy%u(QLBc2<dJa{RP@?J+ap##u#u;Y4}jSK39*fgX-m-a?9Bu}a?;SRI<aY5`Tad0Yd3_C8b&2OX^a(&Ny0ufDU7""Xqz7GcmKxmx<<>)3gO?%#KaT3x2L{&;Pg66<<rpd%1#Tb3Q1Bg3&a9^m0VkX8Yk*MnXjmbF6lSB;bLn{>QCJabxXGCwl5if^uNia""R3PvUMc$<Iz(DULo^%;b2Gd)Z*eFS;DK(k9$G)3Yg?bWn+&x}q(v4W_rI?BMXP*f|2~eI+geC~E{}%^?kvVH1VA+Ifv(u-Lvd^>}""CoSq*0?!<ShVDr$&huJvFyEl4v`h@4#S}*-j{MB7`(|5k)b|g21!#1G6prxQUCFfxD0aChxRVF4BoQlS)TR~+Xt3jq1(IdsXB5vn""PM<;w@VEU$=n|x7#yeS=7WhE*g%v%e_d3qFQ;I4Oxbw5d$W*4bAF3?7pp<YwT=2EXQzs4RtyN+ATOOX$oYC+gPTDwQ9@~}sx%K<X""JwVSFG8%A1@dN7RV+K50%B8Tg`sXRK4%oZ<GdEZn(scP1C-iWEuV9ISjqze4z)r<c0^g4E4?%{Nl8&;E!+gU1I&_xS_G>>AR1!5o""3ycfmP+M2ez>JJ^;_+P{C0M*5N?c8u_>jZb`hJBPV3P(jI~#JxQxeNEKbBXR8wi(=y~6Ng4#1cES!-d}QayM|6!nxJyf&?r{KDfy""RPXq-IfgjvF|JK)fd$!ILO6n1d)i3#=Q$!|G6q+A8rwYJ;3v<e7zI@&WkeDAJ~m`Jx1>he$?vo!p%?KDylHOpUxZ+r6r0=(g31e1""oE&-ir`D6CElWV=-U}MOYQil9k@NI%N)3}ze)j8Y3gV%oU8x|1Se|x!PDc!s*CE{pks?jSw};u5hO~tqVNB5{V$bQ>S1?J=eOj|D""U~DlX(ABLiTgiaGR+`L9fA6TPX2|5ZBE3UK$sf+I0J-h4#Y*+P*#0lK`Q}4LS7xUGuMt%J%bEFQi{~<9Po&jOwbQN|BXDF5yze%h""Nh-@4C9y<&YHf_+3IPL3U1bGtnwt;zQ^h6TiSImA<tL%WjE3V_ejuD2s+1ev=hB<+BLG)swOo2>Y!vYdy!6Vu!2)89FReq1nBPPf""^&}k+zMUK*{y{`h^LvmlPBIGTxQO5jNL5y~(Sb!HYVpg?$>Drj6EbE6uz>-Z6ideAMBM)3jZ;zmSIEr~cZXOlZ*nb2r2$~^6GxgM""XgDZo5uzp0s}~4V&WKZB+qk%U8x9ry0Kg)RaTmrS4lztnkk$N7)}LQ|aPI4$7<Ghi?hAr)W0|#jQp19!LXIOxaM}H5H6z|Ly6zXY""&nw&)l#$VY96lVvehf;VR|~RCG8BD9@gPGOaupG^+?QHCwuFOuB%Ssy_HqK3&KISKks{Ei&q<PH&%O+GJ?Cdq=}f&oE-lduky`i(""{-GJNeFv<A=TN0(7CkNu7;Ka$m%$qhU%QH9edZ%G>uHhqX6hh~@J%(;;1g6^P$@RB>aHkN3kxuD$1rHeeIur)n^N1iz5PLJk#mdk""sY=5Kz77*K-BC=fXR6iUVD~x`!D;<mZC7AB5Gv{Ka`ia7MQiBDbkDO85G+|--|RXAYV_S<Nn7zyQy8v3MM3N{wMY*deHSWQ1YA$>""d<e@Zq0;XZ4sx8E3i(Du>m_=j@NljG!V<|izaOJH!gqYjQd(ljf`@0v%It3mIVPY_iJclM`r^FQi?ZyuypN1U)>JNQ=>0UH43{$?""$m(=9%fPuoc+ONbJ+bn*ZCfxpUK16g*tkVa+<Lb>i;H#05Uh8Ev|*oSyE*~t?6GSI%}PPn(=VL;wWID}My>Ou=rol&Vq|bdSJ?vm""0by(Qtb<E+XD=8ldn)~6;l<RZ$7S_bN<F=^ScOUXcb^?GY*^Mi-Y-<(zY9u*gd6n4PDD!CIh#@Y0R(hAy8JR80x&Riiz{!AL;sOq""&5?kLvh~G)a(#eXIV=J?LMf71Eiub)Tm)5Z9>gdStk7l76EgpaK+g-myv!I0n5rG(*^iYQY{LkI<%?m>qTlip5DrW~%Nx_eiLjJh""6MI7WxE^F|ikF_%UI`<EwX~ymBOrFZ2L1f~dVO_#Ll^mOL>A>zwFzu6${0K3Q*MK*Th5DvU~Jr6LH=d<6iQKYIU2}Axl^!#Wc*8%""yyh0sfT~xRCstVtQ=5@!-rF9_V6?i$E_rtp|4Utt5?8(hRUnLr28V!mF$rT4@*tSRx&5E~^d6V?%f;Ka;%DODtP7x45yCHGSnqz7""@QX0aj`5?G$Id=2w(kD!girx<f~tJpqjNqHpN!*T8D2wN{=vYNcjiW0=4R-_Y)_C_?!$D7!;F}K{z^F8WDsbG`er#RT|e3(sIB}6""b1vH3R(DlY8A{}Y@mjg>rB{I=;i7W7;>k^A3tom}8`d^!q#>t$p&0_Dk&;{bH4C^H#Hj(APIIjP?VY!up(*vwF#6*OY}m+TiR5Ox""Nn!p!By~Iq)N{mZS!rU2Xd8gEbby4=sl_+g0p>Vct~Pj)z4Pz6a4bJx_`d#q!;C+o@Keiz1*59Ea0Av=7VI#TEd)wUPRh3-_$uHA""m8bv!v{aY|mz4W(00GZwzJ~(<?+P=8vBYQl0ssI200dcD")))
Loading