Non-record Submission: Text Diffusion + Retrodiction + TTT + Depth Recurrence

[akaiHuang]

Summary

This PR adds a non-record 16MB submission under records/track_non_record_16mb/2026-04-02_Meadow_TextDiffusion_Retrodiction_TTT_DepthRecurrence.

Techniques included:

• Text Diffusion (CDM) with Sequential Unmasking eval
• AR Retrodiction
• Test-Time Training (full-model AdamW)
• Depth recurrence experiments
• Custom v4096 tokenizer

Files

• README.md
• submission.json
• train_gpt.py
• train_cdm.py
• eval_sequential_unmasking.py
• eval_ttt.py
• bpe_v4096.model
• train.log

This is intended for the non-record track.

[akaiHuang]

Update

This PR is for the non-record track.

Current results:

• AR + Retrodiction (v4096): 1.497
• AR + TTT (full-model AdamW): 1.492
• Shared AR+CDM (single model): 1.503 (~2.3MB)
• CDM + Sequential Unmasking: 2.570

Some exploratory H100 coarse-to-fine entries in the README are marked “no log saved” and are research observations, not record claims.

Next: run 3–5 seed reproducibility on 8xH100 and optimize for strict 10-minute train/eval constraints.

[akaiHuang]

v3.5 update pushed to the PR head branch.

Main changes in this revision:

• Canonical README.md is now the v3.5 writeup; reviewers no longer need to open README_v3_5_DRAFT.md.
• The 11L headline now uses the true final training step with 5 fresh shared-model seeds and 1 fresh matched causal-only control seed.
• Main method-level result at 11L: shared CF 1.3009 ± 0.005 vs matched causal-only final-checkpoint control 1.3214, for a 5-seed mean delta of -0.0205 BPB.
• The single best shared seed (SEED=1337) is retained only as a post-hoc deployable-artifact reference: 1.2924, i.e. -0.0290 BPB versus the matched control.
• The original 6-run 5L+11L sweep is retained as §3.2 cross-scale evidence, while legacy intermediate-checkpoint 11L numbers are moved to Appendix A for traceability with v3.3.
• Supporting scripts are synchronized with the v3.5 methodology:
  • train_cdm.py now writes step_final.pt
  • train_ablation_runner.py supports --seed for the per-seed reruns in seeds_run/
  • eval_cf_ablation.py can load final-state .npz saves directly

This remains a non-record submission. The 11L rows are not claimed as record candidates; they are filed under the non-record track explicitly.

Standalone research diary mirror: https://github.com/akaiHuang/meadow-golf

[MatoTeziTanka]

Community Review — Non-record Submission: Text Diffusion + Retrodiction + TTT + Depth Recurrence

Compliance: NEEDS AUTHOR ACTION — train_gpt.py fails to import on CT2038 (Python 3.10 / torch 2.10.0+cpu)

What I found: The CPU smoke test on CT2038 (proteus-engine, 128 GB RAM, Triton 3.6.0, flash_attn stub, cutlass_evt_fusion stub) failed at the import step with:

ModuleNotFoundError: No module named 'mlx'

A few of the common patterns I've seen for this class of error in the 2026-04-11 sweep:

• PEP 701 f-string nesting — e.g. log(f" {cat}: {", ".join(...)}") is valid Python 3.12+ but invalid Python 3.10 because the inner ", " re-enters the outer double-quote context. One-character fix: ', ' instead of ", ". See PR Record: SP8192 + Improved Parallel Residuals + Muon 0.97 + LR 0.03 + Legal TTT — val_bpb 1.07785 (3-seed mean) #1541 / Record: SP8192 + Triple Recurrence + Banking + Fused MLP + Muon 0.97 — val_bpb 1.0778 (3-seed mean) #1523 for reference.
• Missing flash_attn variants — e.g. from flash_attn_interface import flash_attn_varlen_func when the wrapper script only stubs flash_attn_func. Not a PR defect on H100s, but the eval image / CPU preflight path needs a guarded import.
• Local compiled extension — e.g. import cutlass_evt_fusion from a records/*/cutlass_evt_fusion/ subfolder that isn't on the import path at smoke time. Usually an import-order issue inside the script.
• Actual syntax error — typo, missing bracket, etc.

Recommendation: Could you run python3 -c "import py_compile; py_compile.compile('train_gpt.py')" on your records-folder train_gpt.py under Python 3.10 specifically? The eval image is Python 3.10 per Issue #17 / the README, so any parse error on 3.10 blocks the submission at import time before any of the scored-eval logic runs.

Once the parse/import issue is fixed, I'll re-run the compliance audit through the normal pipeline. No other flags identified yet because the audit halts at the import step.

---

Reviewed by @MatoTeziTanka — The Agora. CPU smoke test (CT2038 proteus-engine, 2026-04-11): IMPORT_FAIL — ModuleNotFoundError: No module named 'mlx'. Classification via classify_prs.py AST-based classifier; full compliance audit deferred until the import issue is resolved. Auto-drafted from a template and spot-checked before posting.

[akaiHuang]

Thanks for the smoke-test details and for catching this before the official audit.

The four files that triggered the import error (eval_cf_dualbrain.py, eval_ttt.py, eval_sequential_unmasking.py, leakage_test.py) are Apple-Silicon-only pre-flight artifacts referenced in §3.3 / §3.4 / §3.6 of the README. They are not part of the H100 production training or evaluation path (which is train_cdm.py + eval_cf_ablation.py), but they live in the same PR folder so your folder-level smoke test correctly hit them.

Fix pushed in commit a4286a4: each file now wraps its import mlx in a try / except ImportError block, installs a minimal stub class on failure (so module-level definitions like class Foo(nn.Module): and COMPUTE_DTYPE = mx.bfloat16 still parse), and sets _HAS_MLX = False. leakage_test.py additionally moves its module-level test logic into a _main() function gated by if __name__ == \"__main__\": so that importing the file is a no-op.

Locally verified:

• py_compile passes on all four files under Python 3.10 syntax.
• Import succeeds on a real Apple Silicon machine with MLX installed (_HAS_MLX=True, original behaviour preserved).
• Import succeeds on a simulated mlx-blocked environment via a custom `import` override (_HAS_MLX=False, stub path).

No README, no scoring path, no numbers change. Please re-run the audit at your convenience — happy to help debug if anything else trips.

[MatoTeziTanka]

Re-audited at head SHA a4286a4.

Fix confirmed. The four MLX-dependent files (eval_cf_dualbrain.py, eval_ttt.py, eval_sequential_unmasking.py, leakage_test.py) all now wrap import mlx in try/except ImportError with _HAS_MLX = False fallback and minimal stub classes. leakage_test.py moves its module-level logic into a _main() gated by if __name__ == "__main__". All files compile clean under Python 3.10.

As you noted, these are Apple Silicon pre-flight artifacts — not part of the H100 scored eval path (train_cdm.py + eval_cf_ablation.py). The fix is the right approach: guarded import so folder-level smoke tests don't false-positive while preserving full functionality on actual Apple Silicon machines.

The train_gpt.py (H100 path) compiles clean. I'll queue the full compliance audit on the active eval path for the next sweep.

---

Re-audit by @MatoTeziTanka. Verified MLX import guards in all 4 files, py_compile OK under Python 3.10.