Record: 11L XSA-all + 7-gram cache (mean val_bpb=1.0465)

[hypery11]

Results

Seed	val_bpb
42	1.0467
1337	1.0470
2024	1.0457
Mean	1.0465
Std	0.0007

• Artifact: 13.99 MB
• Train: 600s on 8xH100 SXM
• Eval: ~116s

Method

11-layer transformer with XSA-all (Exclusive Self-Attention on all layers), LeakyReLU(0.5)^2, Value Residual, Gated Attention, BigramHash(10240), SmearGate. GPTQ-lite int6 + zstd-22. EMA(0.997) + Tight SWA + Late QAT.

7-gram backward-looking eval cache (alpha=0.40, 4M buckets). Score-first, deterministic, no TTT.

Architecture builds on community techniques from PRs #609, #549.

• 8xH100 SXM, train ≤600s
• Eval ≤600s (116s)
• Artifact ≤16MB (13.99MB)
• 3-seed validation (std 0.0007)

[MatoTeziTanka]

Community Review — Record: 11L XSA-all + 7-gram cache (mean val_bpb=1.0465)

BPB: 1.0465 | Compliance: FLAG — hashed n-gram cache with target-in-key (PR #779 family pattern)

What I found in the code (head SHA 5ed06ab2129f, file records/track_10min_16mb/2026-03-25_11L_XSA_7gram/train_gpt.py):

The n-gram lookup key at line 1143 is constructed by XOR-ing the target token into the hash:

line 1143: full_key = <hash> ^ (tgt_np * ng_primes[...]) & mask

This matches the full_key = ((ctx_hash ^ (target * primes[k])) & mask) construction that @valerio-oai ruled disallowed on PR #779 (comment 4145781641, 2026-03-27). Per the mechanism explanation, hashing the target token into the lookup key only reweights the correct token — in the hash-collision limit this drives P(correct) → 1 regardless of the data, which inflates the reported BPB without producing real compression.

Per Issue #1017 condition 1, p_t may depend only on the artifact and x_1...x_{t-1}. Because the lookup key at line 1143 is a function of the target token, the count read at scoring position t depends on x_t itself — which is the core violation the #779 ruling targets.

Cluster context: this same structural pattern has been closed on 15+ PRs under the #779 ruling as of 2026-04-11 (#779 itself, #770, #798, #808, #825, #786, #797, #909, #940, #761, #776, #788, #774, #778, #715, #758, #702 upstream, #1488). The base neural model is unaffected by this flag — in every case where the authors resubmitted without the n-gram cache, the base val_bpb has been in the ~1.10-1.15 range (standard for the SP1024 11L class).

CPU smoke test (CT2038 proteus-engine, 2026-04-11): import OK in 0.10s, dim=512, layers=11, vocab=1024, code=85725 B, SMOKE_TEST_PASS

Verdict: COMPLIANCE FLAG — target-in-key hashed n-gram cache, same family as PR #779.

Recommendation to @cocohearts @valerio-oai @0hq @yuzhougu-oai @notapplica: CLOSE under the same ruling as the rest of the family-bug cluster. A context-only resubmission (drop the target from the lookup key and use a full-vocabulary reweighting from a single context row, per @valerio-oai's suggested legal path on #779) would be welcomed.

---

Reviewed by @MatoTeziTanka — The Agora. CPU smoke test (CT2038 proteus-engine, 2026-04-11): import OK in 0.10s, dim=512, layers=11, vocab=1024, code=85725 B, SMOKE_TEST_PASS. Classification via deterministic AST-based classify_prs.py (pattern bank derived from ~65 manually-reviewed PRs earlier in the 2026-04-11 sweep). This review was auto-drafted from a template and spot-checked before posting — if the template misread your code, please call it out so I can iterate the classifier.