Skip to content

docs: consolidate kernel roadmap, refresh architecture + crate READMEs - #284

Closed
ekryski wants to merge 6 commits into
0xClandestine:devfrom
thewafflehaus:ek/docs-consolidation
Closed

docs: consolidate kernel roadmap, refresh architecture + crate READMEs#284
ekryski wants to merge 6 commits into
0xClandestine:devfrom
thewafflehaus:ek/docs-consolidation

Conversation

@ekryski

@ekryski ekryski commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

What this is

Docs-only. No code changes — consolidates the kernel-organization docs into one reference, brings the architecture/crate docs up to date with the current (multi-backend, subprocess-runner, convolution/-consolidated) codebase, and removes a pile of redundancy.

Kernel docs: 3 → 1 + a relocated style guide

The kernel-organization story was spread across three overlapping docs with heavy duplication (file skeleton, variants, primitives, and block-scaled patterns each appeared 2–4×). Split cleanly by purpose:

  • docs/specs/KERNEL_CONSOLIDATION_PLAN.md (new, the singular roadmap) ← KERNEL_ORGANIZATION.md + CONV_CONSOLIDATION_PLAN.md + the structural parts of CONV_KERNEL_PATTERNS.md. Covers the kernels/<family>/ target layout (dropping the mlx/ + ffai/ split), the three LOC-reduction tools (shared primitives → #[kernel(variants(...))] → op×format merge), the proven convolution/ exemplar, and the family-by-family migration order.
  • docs/STYLE_GUIDE.md (moved from crates/metaltile-std/, the authoring authority) ← the existing style guide + CONV_KERNEL_PATTERNS.md's authoring content. De-mlx/ffai'd, the removed #[bench] name key fixed, a shared-primitives section added. docs/developing.md now links to it.
  • Deleted: KERNEL_ORGANIZATION.md, CONV_CONSOLIDATION_PLAN.md, CONV_KERNEL_PATTERNS.md, and the duplicate convolution/PATTERNS.md + PLAN.md.

Roadmap (where files go) vs authoring (how to write one kernel) are now one doc each.

ARCHITECTURE.md updated to today's codebase

  • Multi-backend codegen — the CodegenBackend seam (Target = Metal/Cuda/Hip/Spirv, TargetProfile, MmaStrategy, the msl/cuda/hip/spirv generators); Metal default, CUDA/HIP/Vulkan feature-gated and validated by the GPU-vs-GPU reference corpora.
  • Subprocess execution — the runner is no longer in-process; tile spawns __tile_runner and streams ProtocolMessages. Rewrote that section + the bench/test diagrams.
  • metaltile-std modules (mlx/ffai/convolution/quant), the command set (clean/config, emit = build --emit, --backend), and the renamed metric path.

Crate READMEs brought current

All seven crates/*/README.md audited and fixed: codegen/runtime/facade no longer "Metal only"; metaltile-core's GpuFamily is in protocol.rs; metaltile-macros paths corrected + the dead #[kernel(bench(op=,class=,mlx=))] system replaced with #[bench]/#[test_kernel]/variants; metaltile-std rewritten as the kernel stdlib (not "benchmark metadata"); metaltile-cli gains the missing subcommands + --backend.

Also

  • Moves the temporary hip_vulkan_phase* / cuda_bench_report markdown into docs/specs/.

This removes far more than it adds — +318 / −2866 lines, mostly redundancy. All internal doc links verified to resolve.

ekryski added 5 commits June 12, 2026 05:59
These two planning docs under crates/metaltile-std/src/convolution/ are
duplicates of the canonical specs in docs/specs/:

  convolution/PATTERNS.md  ->  docs/specs/CONV_KERNEL_PATTERNS.md
  convolution/PLAN.md      ->  docs/specs/CONV_CONSOLIDATION_PLAN.md

Keep the docs/specs/ copies as the single source of truth; nothing references
the in-tree duplicates.
The doc described the pre-#275 single-backend, in-process-runner design. Bring
it current:

- Multi-backend codegen: the CodegenBackend seam (Target = Metal/Cuda/Hip/Spirv),
  TargetProfile (lane width, MmaStrategy), and the msl/cuda/hip/spirv generators.
  Metal is the default/mature backend; CUDA/HIP/Vulkan device execution is
  feature-gated and validated by the GPU-vs-GPU reference corpora.
- Subprocess execution: the runner is no longer in-process — `tile` spawns
  `__tile_runner` (project-linked, inventory populated) and streams ProtocolMessage
  JSON lines. Rewrote the "subprocess readiness" section accordingly.
- metaltile-std modules: mlx / ffai / convolution / quant (was mlx + ffai only).
- Commands: added clean + config; clarified `emit` is `build --emit
  msl|metallib|swift|ir|all` and that --backend applies to bench/test only.
- Fixed companion-doc links (../cli.md, ../developing.md) and the now-renamed
  metric path (to_gflops / estimate_profile / classify_bottleneck /
  device_specs::lookup → ProfileInfo; was derive_metrics / DerivedMetrics).
Three overlapping docs (KERNEL_ORGANIZATION, CONV_CONSOLIDATION_PLAN,
CONV_KERNEL_PATTERNS) collapse into one singular reference,
docs/specs/KERNEL_CONSOLIDATION_PLAN.md — the kernels/<family>/ target layout
(no more mlx/ + ffai/), the three LOC-reduction tools (shared primitives,
#[kernel(variants)], op×format merge), the proven conv/ exemplar, and the
family-by-family migration order.

STYLE_GUIDE.md moves crates/metaltile-std/ → docs/ and becomes the authoring
authority ("how we want kernels written post-consolidation"): de-mlx/ffai'd,
the removed #[bench] name key fixed, a shared-primitives section added, family
mod.rs paths. developing.md's kernel-writing-philosophy now links to it; the
authoring patterns from CONV_KERNEL_PATTERNS fold in here rather than the plan.

Also fixes the stale ../specs/ links in docs/README.md (specs live at
docs/specs/), repoints ARCHITECTURE.md, and indexes both new docs.

Roadmap (where files go) vs authoring (how to write one kernel) are now one doc
each, with the redundancy across the old four removed.
The crate READMEs had drifted across several big changes. Fixes:

- metaltile-codegen / -runtime / metaltile (facade): no longer "Metal only".
  Codegen is multi-backend (CodegenBackend, Target = Metal/Cuda/Hip/Spirv,
  TargetProfile, the cuda/hip/spirv generators); runtime owns device/ (Metal +
  feature-gated cuda/hip/vulkan), dispatch/, cache/; the facade hosts harness/ +
  runner/ and runs GPU work in the __tile_runner subprocess.
- metaltile-core: GpuFamily lives in src/protocol.rs (not src/gpu_family.rs);
  documents the protocol wire types.
- metaltile-macros: corrected module paths (kernel/body.rs, kernel/sig.rs,
  bench.rs, derive/mod.rs), added kernel/variants.rs, and replaced the dead
  #[kernel(bench(op=,class=,mlx=))] system with the current #[bench] /
  #[test_kernel] / #[kernel(variants(...))].
- metaltile-std: rewritten — it's the kernel stdlib (mlx/ ffai/ convolution/
  quant/ probe/ utils), not "benchmark metadata"; dropped the removed
  BenchSpec/bench_types/run_spec framing.
- metaltile-cli: added the missing subcommands (test, clean, config, init,
  update, completions) and the --backend flag.

Cross-doc links point at the relocated style guide / consolidation plan.
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Jun 12, 2026
…uestions

- BENCH_METRICS_SPEC.md: mark the metrics work + Appendix B precision support as
  implemented/complete; note M5 fp8/fp4 acceleration arrives with Metal 4.1;
  trim the stale capture context.
- KERNEL_CONSOLIDATION_PLAN.md: turn §9 open questions into decisions —
  (1) rename ffai_/model-prefixed kernels to mt_<op> (the FFAI emit consumer is
  regenerated, so names are not pinned); (2) keep vision/ + audio/ folders;
  (3) f16-scale twins fold into a ScaleKind format axis, not separate kernels;
  (4) kernels that mirror an upstream reference keep their .with_reference()
  comparator indefinitely. Reconciled §6 (rename step), §7 (scale axis), and §8
  (inventory names do change; emit is regenerated).
@ekryski ekryski closed this Jun 12, 2026
@ekryski ekryski reopened this Jun 12, 2026
@TheTom

TheTom commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Strong consolidation. I went through each doc against the current code; notes below.

What's implemented vs. aspirational, doc by doc. ARCHITECTURE.md is accurate to today's codebase: the subprocess runner is real (ProjectRunner spawns __tile_runner and parses ProtocolMessage lines, landed in #249 and refined in #260), --backend cuda|hip|vulkan routes through the feature-gated devices exactly as described, and all four referenced corpus files exist, vulkan_sdpa_multi.rs included. KERNEL_CONSOLIDATION_PLAN.md is correctly framed as a roadmap, with two spots where target state reads as achieved: the wave table marks conv/ done while §4 lists open phases for it (remaining macro_rules!, unmerged _block_scaled/_mma files), and the ~1,600 LOC figure in the Before/After table is a target, worth labeling as such. The style guide is the one doc that lags both directions, see below.

Style guide content didn't make it into the diff. docs/STYLE_GUIDE.md is a 100% similarity rename, but the body describes content changes (de-mlx/ffai, #[bench] name-key fix, new shared-primitives section). As it stands the guide still routes new kernels into mlx/ vs ffai/ (§1) and blesses the ffai_ prefix (§3), both of which the plan removes, and it teaches #[bench(name = "...")] in §2/§5/§7/§10, which the macro no longer accepts (bench.rs doc: the bench name is always the function name), so the worked examples won't compile. The plan's §3 also points to "STYLE_GUIDE §5–6" for shared-primitives authoring detail, but no doc has that section yet.

Kernel registry could use a paragraph in ARCHITECTURE.md while we're here. Registration spans three crates: metaltile-core re-exports inventory for macro-expanded submit! calls, metaltile-codegen owns KernelEntry/all_kernels() (placed next to KernelInlinePass, its only consumer), and the facade's harness/registry.rs holds the bench/test registries. The __tile_runner bin in metaltile-std exists largely to extern crate metaltile_std; so the linker keeps the inventory statics, and tile init scaffolds a per-project copy. All working, but the linker-force line is load-bearing and easy to "clean up" by accident, and the three-crate split isn't obvious from the doc. Worth capturing the why, not just the what.

Leftover dead-system refs. crates/metaltile/README.md:100,188 and crates/metaltile-macros/README.md:216 still document #[kernel(bench(...))]. Body line count is also stale (+476/-3240 now).

Two suggested additions. (1) §9.1 calls the ffai_mt_ rename non-breaking, but downstream consumers reference the generated Swift symbols directly, so each family migration needs a paired consumer-side PR; worth a sentence. (2) A short note on sequencing the family git mv waves around in-flight kernel branches would save everyone rebase pain.

Happy to take any of these as follow-ups if you'd rather land the bulk now.

@ekryski

ekryski commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator Author

As per @0xClandestine's request, as relayed to me by @TheTom, closing in favour of moving rapid iteration to forks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants