Skip to content

refactor(sampling): migrate sampling family into kernels/sampling/ - #17

Merged
TheTom merged 1 commit into
devfrom
ek/kernel-reorg-sampling
Jun 22, 2026
Merged

refactor(sampling): migrate sampling family into kernels/sampling/#17
TheTom merged 1 commit into
devfrom
ek/kernel-reorg-sampling

Conversation

@ekryski

@ekryski ekryski commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

What

Wave-1 kernel-family port: moves the sampling family (the logits→token
pipeline) out of the legacy mlx/ + ffai/ split into kernels/sampling/ per
docs/specs/KERNEL_CONSOLIDATION_PLAN.md, and adds the mt_ prefix to the
previously-unprefixed ffai/ kernels.

Changes

  • mlx/{softmax,sort}kernels/sampling/ (mt_softmax, mt_sort/mt_merge/mt_sort_segmented).
  • ffai/sampling.rskernels/sampling/categorical_sample.rs; softmax_categorical_sample
    mt_softmax_categorical_sample (file renamed off the family name to avoid
    module_inception).
  • ffai/logits_{topk,top_p,min_p,processors}kernels/sampling/ with mt_
    prefixes (mt_logits_topk_mask, mt_logits_top_p_mask, mt_logits_min_p_mask,
    mt_logits_temperature, mt_logits_repetition_penalty).
  • Excluded from this family: dsv4_*_topk (moe/sdpa expert/indexer routing) and
    upsample_nearest1d (audio) — they go with their own families.
  • Updated the categorical-sample integration-test imports, cross-ref doc comments
    (dsv4_indexer_topk, the SPIR-V codegen note), the plan (sampling marked done),
    and the KERNEL_AUDIT rows.

Generated MSL per kernel is unchanged — only module paths and inventory names.

Verification

  • cargo build -p metaltile-std + cargo clippy -p metaltile-std --tests clean.
  • All sampling kernel tests pass via tile test (softmax, sort, the four logits
    masks/processors, categorical sampler).
  • softmax_categorical_sample_matrix integration test passes.

Note on stacking

Stacked on the norm PR (#16) → conv+rope (#15) → docs (#14); the diff includes
those until they merge. Each family lands as its own PR and consolidates cleanly.

@github-actions github-actions Bot added the ignore-for-release Hidden from release notes label Jun 12, 2026
@ekryski
ekryski force-pushed the ek/kernel-reorg-sampling branch 2 times, most recently from 8bcb1e9 to dc6e2f0 Compare June 12, 2026 23:21
@TheTom

TheTom commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Good consolidation: the five logits_* modules folding into kernels/sampling/ with mt_ names matches the plan's one-file-per-op-family-member rule, and the simple elementwise kernels (mt_logits_temperature, mt_logits_repetition_penalty) follow the generic <T> + f32-accumulate house pattern. No issues found in the increment over #16. The KERNEL_AUDIT.md sync in every PR of this stack is appreciated, it keeps the plan document honest as the waves land.

@TheTom

TheTom commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

@ekryski clean. Nice catch repointing the dangling spirv doc cross-ref (crate::mlx::sort::mt_sortkernels::sampling::sort). One conform heads-up for consumers: ffai_logits_{topk,top_p,min_p} gained a _mask suffix (mt_logits_*_mask). Approve.

Moves the logits->token pipeline out of the legacy mlx/ + ffai/ split into
kernels/sampling/ and adds the mt_ prefix to the unprefixed ffai kernels:

- mlx/{softmax,sort} -> kernels/sampling/
- ffai/sampling.rs -> kernels/sampling/categorical_sample.rs
  (softmax_categorical_sample -> mt_softmax_categorical_sample; renamed the
  file off the family name to avoid module inception)
- ffai/logits_{topk,top_p,min_p,processors} -> kernels/sampling/, with mt_
  prefixes (mt_logits_topk_mask, mt_logits_top_p_mask, mt_logits_min_p_mask,
  mt_logits_temperature, mt_logits_repetition_penalty)

Generated MSL per kernel is unchanged; only paths + inventory names change.
Updated the softmax_categorical_sample integration-test imports, cross-ref
doc comments (dsv4_indexer_topk, spirv codegen), the plan (sampling done),
and KERNEL_AUDIT rows.

cargo build + clippy clean; all sampling kernel tests + the categorical-sample
integration matrix pass.
@TheTom
TheTom force-pushed the ek/kernel-reorg-sampling branch from dc6e2f0 to 34d6904 Compare June 22, 2026 21:23
@TheTom
TheTom merged commit 8bc9d9b into dev Jun 22, 2026
8 checks passed
@ekryski
ekryski deleted the ek/kernel-reorg-sampling branch July 13, 2026 20:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ignore-for-release Hidden from release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants