Skip to content

Title: Record submission: FarnsworthEngine v1 — val_bpb=1.1303 (mean 1.1313, 3 seeds) #270

@timowhite88

Description

@timowhite88

Title: Record submission: FarnsworthEngine v1 — val_bpb=1.1303 (mean 1.1313, 3 seeds)

Body:


Requesting leaderboard addition for PR #254.

FarnsworthEngine v1 — TTT + 11L Int6 MLP3x + SmearGate + BigramHash + OrthoInit + Muon WD + SWA + FA3 + Sliding Window

┌──────┬─────────┬──────────┐
│ Seed │ val_bpb │ val_loss │
├──────┼─────────┼──────────┤
│ 1337 │ 1.1303 │ 1.9085 │
├──────┼─────────┼──────────┤
│ 42 │ 1.1312 │ 1.9100 │
├──────┼─────────┼──────────┤
│ 7 │ 1.1323 │ 1.9118 │
├──────┼─────────┼──────────┤
│ Mean │ 1.1313 │ 1.9101 │
└──────┴─────────┴──────────┘

  • Artifact: 15,877,181 bytes (under 16,000,000)
  • Training: 600s, 7,248 steps, 81.5ms/step on 8xH100
  • Eval: 129s (43s TTT + 86s sliding window stride=64)
  • 3 seed logs included in PR
  • Beats current merged SOTA (1.1748) by 0.0435 BPB

PR: #254

@0hq

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions