Skip to content

Add ConsensusWindow Bypass (FAT-Golf) non-record submission#919

Open
TheDryhtscipe wants to merge 1 commit intoopenai:mainfrom
TheDryhtscipe:consensus-window-bypass
Open

Add ConsensusWindow Bypass (FAT-Golf) non-record submission#919
TheDryhtscipe wants to merge 1 commit intoopenai:mainfrom
TheDryhtscipe:consensus-window-bypass

Conversation

@TheDryhtscipe
Copy link
Copy Markdown

Features depthwise causal convolution bypass from my own local ORC FAT-AR architecture. Small-scale results: -0.075 BPB post-quantization across 3 seeds. H100 compute needed for full-scale validation.

Depthwise causal convolution bypass from ORC FAT-AR architecture.
Small-scale results: -0.075 BPB post-quantization across 3 seeds.
Awaiting H100 compute for full-scale validation.
@MatoTeziTanka
Copy link
Copy Markdown

Community Review — Add ConsensusWindow Bypass (FAT-Golf) non-record submission

BPB: 0.075 (cache parse — may be delta/std, not val_bpb; check PR title) | Compliance: LOOKS CLEAN — score-first-per-chunk TTT (legal #1416/#1423 pattern)

What I found in the code (head SHA ed85cee767dc, file records/track_non_record_16mb/2026-03-27_ConsensusWindow_Bypass_FAT_Golf/train_gpt.py):

The TTT path at line 1165 implements the score-first-per-chunk pattern: each chunk is scored under torch.no_grad() / inference_mode() before the base_model.train() + SGD adaptation runs on that same chunk, with an is_last_chunk guard so the final chunk gets no adaptation pass. This is the structural shape the legal frontier uses (PRs #1416 erichroepke, #1423 aryanbhosale).

Per Issue #402 and Issue #677, TTT is legal when each token is scored before the adapter updates on it, and that's what the code does here — chunk ci is scored under weights adapted only on chunks 0..ci-1. No prequant_ttt_adapt_adamw(val_tokens, ...) multi-epoch fine-tune, no scored-region SLOT, no target-in-key n-gram cache.

CPU smoke test (CT2038 proteus-engine, 2026-04-11): import OK in 0.07s, dim=512, layers=11, vocab=1024, code=95276 B, SMOKE_TEST_PASS

Verdict: LOOKS CLEAN.

Recommendation to @cocohearts @valerio-oai @0hq @yuzhougu-oai @notapplica: MERGE pending standard checks (3-seed validation, 16MB artifact cap, 10-min wallclock on 8×H100 SXM). The compliance picture matches the legal reference frontier and no flags were raised by the classification pass.

Auto-classification caveat: this review was drafted by the AST-based classifier against a template derived from manually-reviewed cluster PRs (#1420, #1450, #1487, #1541, #1529, #1533, #1518). If I've misread a subtlety in your eval path — e.g., multi-epoch TTT that I mistook for single-pass, or a target-in-key lookup I missed in a helper function — please flag it and I'll re-run the audit manually.


Reviewed by @MatoTeziTankaThe Agora. CPU smoke test (CT2038 proteus-engine, 2026-04-11): import OK in 0.07s, dim=512, layers=11, vocab=1024, code=95276 B, SMOKE_TEST_PASS. Classification via deterministic AST-based classify_prs.py (pattern bank derived from ~65 manually-reviewed PRs earlier in the 2026-04-11 sweep). This review was auto-drafted from a template and spot-checked before posting — if the template misread your code, please call it out so I can iterate the classifier.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants