Record: SP4096 + 3-Layer Recurrence + GPTQ Embeddings + SDClip + ETLB — val_bpb 1.0913 (3-seed mean)

[bigbag]

Summary

• val_bpb = 1.0913 (3-seed mean, std 0.0012) | ~14.75 MB | 8×H100 SXM
• SP4096 with GPTQ on embeddings, SDClip, 3-layer depth recurrence, eval-time logit bias
• No SLOT, no TTT — fully compliant

3-Seed Results

Seed	Sliding BPP	ETLB BPP	Artifact
42	1.0901	1.0900	14,748,056
314	1.0920	1.0919	14,744,841
999	1.0922	1.0921	14,746,245
Mean	1.0914	1.0913	14,746,381

Key Techniques

1. GPTQ on Embeddings (int8) — PR Record: SP8192 + GPTQ Embeddings + Depth Recurrence + MuonEq-R + SDClip — val_bpb 1.08563 (5 seed mean) #1394 @clarkkev
2. SDClip — std-dev based quantization (PR Record: SP8192 + GPTQ Embeddings + Depth Recurrence + MuonEq-R + SDClip — val_bpb 1.08563 (5 seed mean) #1394)
3. 3-Layer Depth Recurrence (layers 3,4,5) — PR Record: MuonEq-R + 3-Layer Recurrence + WD=0.095 + MLR=0.022 + All-Int6 — val_bpb 1.0900 (3-seed mean) #1331 @dexhunter
4. ETLB: Eval-Time Logit Bias — PR Record: Pre-Quant TTT + ETLB: Eval-Time Logit Bias for Neural Language Model Compression 1.0898 BPB on PR #1285 base #1399 @resouer
5. QK-Gain 5.0 — PR Non Record: MuonEq-R + Context-Only SLOT + QK_GAIN=5.0 — val_bpb 1.1027 (3-seed mean) #1217 @bigbag
6. WD=0.095 + MLR=0.022 — PR Record: MuonEq-R + 3-Layer Recurrence + WD=0.095 + MLR=0.022 + All-Int6 — val_bpb 1.0900 (3-seed mean) #1331 @dexhunter
7. MuonEq-R — PR Record: MuonEq-R + Depth Recurrence + Mixed Int5/Int6 GPTQ — val_bpb 1.0929 (3-seed mean) #1260 @dexhunter
8. LZMA code wrapper — 18KB code saves ~40KB artifact

Credits

PR #1394 @clarkkev, PR #1399 @resouer, PR #1331 @dexhunter, PR #1217 @bigbag, PR #1260 @dexhunter, PR #1204 @msisovic

Test plan

• 3-seed validation (42, 314, 999)
• All artifacts under 16,000,000 bytes
• No SLOT, no TTT
• Seed 42 reproduced twice (1.0900 both times)

🤖 Generated with Claude Code