ci: restructure into reusable workflows + PR coverage gating - #80
Merged
Conversation
…bench)
Replaces the monolithic `check.yml` + `iron.yml` with focused, reusable
workflows driven by one orchestrator, so each concern is its own file, the
iron binary is built once per run, and adding CUDA/HIP runners later is a
one-line matrix entry.
ci.yml (orchestrator) changes → lint → {build, test, correctness}
release (tag only, needs build+test+correctness)
lint.yml typos, fmt, clippy, commit-hygiene (ubuntu). Every PR + release.
build.yml compile the iron CLI once (cached, uploaded) → `iron build`
kernel compile-check. Consolidates iron.yml's compile+build.
test.yml cargo nextest: Rust logic on ubuntu (GPU tests cfg out there) +
a macOS job for the GPU integration tests, minus the per-kernel
harness (owned by correctness).
correctness.yml `iron test` (GPU-vs-CPU-oracle), parameterised by
runner+backend for future NVIDIA/AMD hosts.
bench.yml full suite (no heavy/light split), manual dispatch only.
Shared setup is DRY'd into composite actions (.github/actions/setup-rust,
setup-iron — the latter also installs the macos-26 Metal Toolchain). Light
path filter skips docs-only PRs for the heavy jobs; lint still runs on them.
Releases are gated: on a `v*` tag ci.yml runs the full suite before calling
release.yml. coverage.yml + pr.yml keep their separate concerns.
- Rename test.yml's ubuntu cargo job `logic` -> `iron` / "Iron Core (ubuntu)". - Make coverage.yml reusable (workflow_call) and wire it into ci.yml so instrumented `llvm-cov --workspace` runs on every code PR (was push-to-main only). codecov now posts the diff-coverage comment on PRs and its status checks can gate merges. - codecov.yml already enforces the requested gates — patch (new/changed code) >= 80% so 0%-covered new code fails, and a per-crate + project ratchet that fails on any coverage decrease. Add a `std` flag for wh-iron-std's host / dispatch code (the coverable half of "kernels"; the `#[kernel]` DSL bodies stay excluded — the proc-macro consumes them so they never execute as Rust, and are validated via GPU correctness / bench instead). To make coverage BLOCK merges, add the codecov "project" and "patch" checks to the branch protection rule. To turn the >= 80% goal into a hard project floor (vs the current no-regression ratchet), flip project `target: auto` -> `target: 80%` once the codebase is at 80% — the first PR coverage run will show where it stands.
…ers, timeouts) Addresses the "no false skips/passes/failures" review concern now that the restructure routes GPU kernel correctness through `iron test` (correctness.yml) instead of the cargo `all_registered_kernel_tests_pass` harness. - `iron test`: with NO filter and zero registered `#[test_kernel]`s, FAIL instead of returning Ok. Zero kernels means the inventory failed to link (dead-strip / registration regression), not that everything passed — a correctness gate that dispatched nothing must never report green. This restores the cargo harness's `total > 0` guard on the CLI path. (A user-supplied filter matching nothing stays a benign warning.) The runner's MPP skip is already precise — it requires BOTH `requires_cooperative_tensors()` AND the `deferred-static-alloca` marker (harness.rs), so no MPP kernel is skipped by a coarse family gate. - Log `sw_vers` on every macOS job (via the setup-iron composite + the standalone Metal Toolchain steps). The coop-tensor skips are an OS-version gap (< 26.5); when the image rolls forward the skips vanish on their own and this makes that visible. (Tom's review rec.) - Bump the macOS GPU-sweep timeouts 30 -> 45 min (correctness / test-gpu / coverage) — headroom as the kernel registry grows, per the review's timeout watch-item. The ubuntu logic job stays at 30.
commit-hygiene is inherently PR-only (it reads the PR title/body/commits and labels/comments the PR). Relocating it alongside the other PR-convention jobs (title lint, label) puts all PR-metadata checks in one workflow and — because pr.yml uses `pull_request_target` — gives it a WRITE token even on fork PRs, so its label + findings comment actually land (they silently no-op under pull_request's read-only fork token today). Safe under pull_request_target: the script reads everything via the `gh` API and the checkout is the BASE ref only (the repo's own script), so no PR HEAD code is executed. lint.yml drops its `pull-requests: write` permission (its remaining typos/fmt/clippy jobs only read), and ci.yml's lint call no longer needs to grant it. Note: the required-check context moves from "Lint / Commits" to "PR Conventions / Commits" — update branch protection accordingly.
…OS, non-PR)
The instrumented `llvm-cov --workspace` on macOS was the single heaviest job
(~19 min: instrumented build + full GPU kernel sweep) and it ran on every PR,
redundantly re-executing the same GPU sweep that correctness.yml already does.
Split it in two:
- coverage-core: ubuntu, `--workspace --exclude wh-iron-std`, no Metal
Toolchain, no GPU. Fast, gates every code PR.
- coverage-kernels: macOS + GPU, `-p wh-iron-std`, NON-PR only, slotted
after correctness. Covers wh-iron-std's kernel host/dispatch paths.
coverage.yml is now reusable/parameterized (runner, cargo-args, metal,
cache-key). Scope the codecov PR patch gate to the core flags so a std-only PR
can't fail patch on carried-forward lines; std stays enforced by its project
ratchet on main. Rename build.yml's `kernels` job to `codegen` to match the
pipeline's mental model. Removes one Metal Toolchain install from every PR.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overhauls GitHub Actions into focused, reusable workflows driven by one orchestrator, adds PR coverage gating, and hardens against false CI results.
Structure
Composite actions (DRY):
setup-rust(toolchain + cache),setup-iron(download the once-built binary + macos-26 Metal Toolchain +sw_vers).Reusable workflows called by
ci.yml:lint.ymlbuild.ymliron buildkernel check. Consolidates the redundant iron.yml compile+buildtest.ymlIron Core (ubuntu)cargo logic + macOS GPU-integration cargo tests (minus the per-kernel harness, owned by correctness)correctness.ymliron testvs CPU oracle — parameterized by runner+backend for future CUDA/HIPcoverage.ymlllvm-cov --workspace→ codecov, on PRsbench.ymlcheck.yml+iron.ymldeleted;coverage.yml/release.ymlmade reusable;pr.ymlunchanged.Behavior
v*tag runs the full suite beforerelease.ymlpublishes.Coverage gating (new)
codecov.ymlenforces patch ≥ 80% on new code (0%-covered new code fails) + per-crate/project ratchet (fails on any decrease). Added astdflag for wh-iron-std host code.#[kernel]DSL bodies stay excluded (proc-macro consumes them; validated via GPU correctness/bench).No false CI results
iron testnow FAILS on no-filter + zero registered kernels ("refusing to report success") — restores the cargo harness's link-regression guard on the CLI path the restructure routes correctness through.requires_cooperative_tensors()AND thedeferred-static-allocamarker) — no coarse family over-skip.sw_verslogged on macOS jobs so the OS-version-gap skips are visible and self-heal when the image advances; GPU-sweep timeouts bumped 30→45.Follow-ups for the owner
target: auto→80%once the codebase reaches 80% (first coverage run shows where it stands).actionlintclean on all workflows + composites.