Add RIOM v1 shared-depth recurrence non-record submission

[hesong0222-dev]

Summary

This draft PR adds a non-record RIOM-style submission under records/track_non_record_16mb/2026-03-23_RIOM_v1_recur.

The core idea is shared-depth recurrence: replace 9 distinct transformer blocks with 3 shared blocks repeated 3 times, with lightweight learned recurrence gates. The tokenizer and dataset remain unchanged from the official SP1024 FineWeb setup, and the change is isolated to the record-local train_gpt.py.

What actually ran

This package is based on a real Apple Silicon MLX development run.

• training steps: 50
• validation prefix: 1,048,576 official validation tokens
• pre-quant: val_loss=5.4143, val_bpb=3.2439
• post-quant: val_loss=5.42207763, val_bpb=3.24862008
• code bytes: 51,404
• compressed model bytes: 2,273,437
• total counted bytes: 2,324,841

Why this is interesting even if non-record

On the same local development prefix, this recurrent package improved val_bpb over the local baseline package while sharply reducing counted artifact size. That is the main RIOM hypothesis in compact form: more effective depth per byte through parameter sharing.

Limitations

• This is not a record claim.
• This is a local MLX development run, not an 8xH100 submission.
• Validation was capped with VAL_MAX_TOKENS; it should be rerun with VAL_MAX_TOKENS=0 before any serious upstream submission.
• The next concrete step is a CUDA/PyTorch port into the official train_gpt.py path.

[MatoTeziTanka]

Community Review — Add RIOM v1 shared-depth recurrence non-record submission

Compliance: NEEDS AUTHOR ACTION — train_gpt.py fails to import on CT2038 (Python 3.10 / torch 2.10.0+cpu)

What I found: The CPU smoke test on CT2038 (proteus-engine, 128 GB RAM, Triton 3.6.0, flash_attn stub, cutlass_evt_fusion stub) failed at the import step with:

ModuleNotFoundError: No module named 'mlx'

A few of the common patterns I've seen for this class of error in the 2026-04-11 sweep:

• PEP 701 f-string nesting — e.g. log(f" {cat}: {", ".join(...)}") is valid Python 3.12+ but invalid Python 3.10 because the inner ", " re-enters the outer double-quote context. One-character fix: ', ' instead of ", ". See PR Record: SP8192 + Improved Parallel Residuals + Muon 0.97 + LR 0.03 + Legal TTT — val_bpb 1.07785 (3-seed mean) #1541 / Record: SP8192 + Triple Recurrence + Banking + Fused MLP + Muon 0.97 — val_bpb 1.0778 (3-seed mean) #1523 for reference.
• Missing flash_attn variants — e.g. from flash_attn_interface import flash_attn_varlen_func when the wrapper script only stubs flash_attn_func. Not a PR defect on H100s, but the eval image / CPU preflight path needs a guarded import.
• Local compiled extension — e.g. import cutlass_evt_fusion from a records/*/cutlass_evt_fusion/ subfolder that isn't on the import path at smoke time. Usually an import-order issue inside the script.
• Actual syntax error — typo, missing bracket, etc.

Recommendation: Could you run python3 -c "import py_compile; py_compile.compile('train_gpt.py')" on your records-folder train_gpt.py under Python 3.10 specifically? The eval image is Python 3.10 per Issue #17 / the README, so any parse error on 3.10 blocks the submission at import time before any of the scored-eval logic runs.

Once the parse/import issue is fixed, I'll re-run the compliance audit through the normal pipeline. No other flags identified yet because the audit halts at the import step.

---

Reviewed by @MatoTeziTanka — The Agora. CPU smoke test (CT2038 proteus-engine, 2026-04-11): IMPORT_FAIL — ModuleNotFoundError: No module named 'mlx'. Classification via classify_prs.py AST-based classifier; full compliance audit deferred until the import issue is resolved. Auto-drafted from a template and spot-checked before posting.

[hesong0222-dev]

Thanks for the smoke test.

I pushed a fix for the import blocker on the RIOM v1 non-record script. The update guards the MLX and sentencepiece imports so the module can be imported on a machine without those packages installed, while still failing clearly at runtime if someone actually tries to run the MLX path without the required dependencies.

What I verified locally after the change:

• python3 -m py_compile records/track_non_record_16mb/2026-03-23_RIOM_v1_recur/train_gpt.py
• direct module import in an environment without mlx and without sentencepiece
• calling main() in that same environment now raises a clear ModuleNotFoundError explaining which runtime dependencies are missing, instead of failing during import

If you re-run the compliance pipeline, the import step should now pass cleanly.

[hesong0222-dev]

Follow-up: I re-ran the checks under Python 3.10 specifically.

• python3.10 -m py_compile records/track_non_record_16mb/2026-03-23_RIOM_v1_recur/train_gpt.py now passes
• direct module import also passes on a machine without mlx and without sentencepiece
• in that same environment, calling main() now fails cleanly with a runtime dependency error instead of failing during import