feat(jit): move the LLVM backend from LLVM 15 to LLVM 21

[AregGevorgyan]

Moves the LLVM JIT backend from LLVM 15 to LLVM 21. Stacked on #310 — review with
git diff integrate/autoresearch-2026-08-19...feat/llvm21-upgrade to see only the LLVM work.

inkwell was never the constraint

inkwell 0.9 — already the pinned version — carries feature flags for LLVM 11 through 22.
The version was one feature string, llvm15-0-prefer-dynamic. Bumping to inkwell 0.10 would
lose a version (it supports 12–22), not gain one.

So the non-CUDA half of this needed zero source changes: it builds and runs on LLVM 18 and
21 as-is. The only thing that genuinely broke is CUDA.

What actually breaks: the CUDA pass pipeline

LLVM 17 removed the legacy PassManager, so --features cuda did not merely warn — it failed
to compile with four no method named add_*_pass errors. run_libdevice_cleanup_passes is
rewritten onto Module::run_passes with the same four passes in the same order.

Verified on hardware (2× RTX 3090, LLVM 21.1.8)

• scripts/verify_cuda_on_gpu.sh: 9 passed, 0 failed, 0 skipped; 19 GPU tests with no
  silent CPU fallback; compute-sanitizer memcheck / racecheck / initcheck / synccheck all
  ERROR SUMMARY: 0.
• PTX diffed against an LLVM 15 build of the same tree, 14-kernel battery (poly, sin, cos,
  tan, exp, log, sqrt, abs, three pow forms, all_funcs, deep_nest, wide): identical set of
  out-of-line libdevice functions in all 14, zero __nv_* and zero tanh survivors in either
  build, 14/14 assemble under ptxas -arch=sm_86, and 896/896 device results bit-for-bit
  identical.
• The PTX does differ, but only in backend codegen — typeless .b64 registers, .ptr .align 1
  param annotations, param loads hoisted above the bounds check, 2–12 % fewer instructions.
  poly, which touches no libdevice at all, shows the same class of diff, which is what
  establishes it as LLVM version noise rather than a pipeline regression.
• Negative control: with the cleanup disabled, all 14 cases fail ptxas on LLVM 21 with
  exactly the documented signature (Parsing error near '.nvvm' on
  llvm.nvvm.tanh.approx.f32, inside __nv_fast_tanhf), and PTX goes 822 KB → 975 bytes when
  it runs. The hazard is fully alive on LLVM 21, and this pipeline is what removes it.

Note nothing in CI builds cuda (cuda_nightly.yml is workflow_dispatch with no registered
runner), so that verification is a manual GPU-box step and this PR does not change that.

CI

Ubuntu 24.04's universe stops at llvm-19-dev, so LLVM 21 comes from apt.llvm.org via a new
.github/actions/setup-llvm composite action — nine jobs across five workflows build jit,
which is why it is an action and not nine copies. It reads the codename from /etc/os-release
so it survives the runner image moving off noble, and it fails loudly at apt-get update if
the pinned suite is ever retired. macOS pins brew install llvm@21; Homebrew's unversioned
llvm is already 22.1.8.

Two job-level LLVM_SYS_211_PREFIX: /usr overrides were removed from release-build.yml:
workflow-file env: beats $GITHUB_ENV, and /usr is wrong for an apt.llvm.org install (its
debs put llvm-config solely at /usr/lib/llvm-21/bin; nothing lands directly in /usr/bin).
Those jobs are release-gated, so this would have passed every PR and failed at the tag.

User-facing consequence

The +jit and +full release wheels move ubuntu-22.04 → ubuntu-24.04, raising those two
wheels' glibc floor from 2.35 to 2.39. They are linux_x86_64-tagged and attached to GitHub
Releases only, never PyPI. The default PyPI wheel is untouched and stays on 22.04. The
attach-optional-linux-wheels artifact glob is updated to match, which would otherwise have
silently stopped matching anything.

cargo semver-checks reports no semver update required — there are no public API changes.

Also here

• AGENTS.md's headline build line no longer carries jit: it needed LLVM dev headers the line
  never mentioned, so a fresh contributor's first command failed on an LLVM error, and
  cranelift already provides a JIT with no system dependencies. This is issue #30a of the
  2026-08-19 autoresearch run.
• The GPU review also caught budget::memory::tests::gmp_accounting_tracks_a_big_rational
  (from fix: the 2026-08-19 autoresearch run — two Critical soundness bugs, a false mathematical claim, and an uncatchable process abort #310) failing 1-in-4 under --workspace: it reads a process-global counter while ~2360
  other tests allocate GMP concurrently. It now samples until one pair is uncontended, with both
  assertions still strict.

Verification

gate	result
cargo test --workspace --features "parallel egraph cranelift jit groebner"	2411 passed, 0 failed (incl. 29 doctests)
--features cuda jit groebner	compiles; GPU-verified above
cargo +1.98.0 clippy — default / cranelift / jit / cuda+jit, -D warnings	clean
cargo doc --workspace with RUSTDOCFLAGS=-D warnings	clean
cargo fmt --check	clean
cargo semver-checks vs origin/main	223/223, no update required
scripts/check_error_codes.py	OK, 217 codes
compile_expr JIT output	matches libm to the last bit

Verified against LLVM 21.1.8 on two independent machines (RHEL 10 llvm-devel here, extracted
apt.llvm.org debs on the GPU box). What neither of us could verify is the packaging path
itself — setup-llvm has never run on a real runner, and brew install llvm@21 is asserted
from the formula API rather than observed. This PR's own CI run is the first real test of both,
and both fail loudly rather than silently.

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 48035af6-294b-41c7-b550-5bf537878ae7

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codspeed-hq]

Merging this PR will not alter performance

✅ 35 untouched benchmarks
⏩ 49 skipped benchmarks¹

---

_{Comparing feat/llvm21-upgrade (e115260) with integrate/autoresearch-2026-08-19 (ea0dee8)}

Footnotes

1. 49 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩