chore - SDK provider preparation rebuilds all ten components on any compiler edit

[dannymeijer]

Area

Tooling (CLI/formatter/test runner)

Summary

Preparing the SDK provider cache takes about 17 minutes, and any change to the compiler checkout triggers the whole thing. Two independent causes, both structural.

The cache key hashes the entire compiler source tree. sdk_provider_store_identity hashes the whole checkout when one is present, excluding only .git, target, and .github. Editing any file invalidates all ten components — including files that cannot affect how a stdlib component compiles, such as src/oven/, src/lsp/, and tests/.

The current design is a deliberate correction of an earlier one, and its own comment says why: "hashing development executable bytes would make identical source checkouts miss after unrelated test builds." Moving from executable bytes to a source hash removed spurious misses from non-reproducible binaries. It over-corrected: now every edit anywhere is a certain miss.

Preparation is all-or-nothing. There is one identity for the whole store and one staging directory. Components have no identity of their own, so a change anywhere rebuilds all ten regardless of what it touched.

The shape that is being thrown away

The components already declare a one-directional dependency graph in sdk-components.toml, and publication_order() already topologically sorts it:

core → {interop, system, async, testing} → {codecs, compression, data} → {observability, web}

Ten components, four levels. That graph is currently used only to order a strictly serial for loop, then discarded.

Giving each component an identity over its own source plus its dependencies' identities makes invalidation follow the graph:

• change stdlib-web source → rebuild web alone, since nothing depends on it
• change stdlib-core → rebuild everything, which is correct, because everything depends on it
• change src/oven/rustc/inspection.rs → rebuild nothing, because no component's inputs changed

This is the same content-addressed reasoning the compiler is adopting for compilation units, applied one level up to component preparation: prove whether an input changed rather than assume it did.

Suggested order

1. Narrow the compiler-source hash to what can affect a component build — the frontend, IR, emission, and provider paths plus the stdlib source. Exclude tests/ explicitly; test code cannot change a component's output, and excluding it directly serves the original motivation quoted above. Cheapest change and it removes most invalidation in practice.
2. Give each component its own identity over its source and its dependencies' identities, and reuse per component rather than per store.
3. Build each dependency level concurrently. The order is already computed; only the loop is serial. Bounded by the critical path core → system → data → web rather than by the component count.

Verification

The failure mode for 1 and 2 is a stale component producing a wrong build, so both need a miss/hit test: edit an excluded file and confirm a hit, edit an included one and confirm a miss, and confirm that changing a leaf component does not rebuild its dependencies while changing core rebuilds everything downstream.

Environment

• Compiler built from 0.6.0-dev.4 at 0a8395835
• Measured while iterating on compiler fixes for the Incan-authored Oven control plane (feature - RFC 119 native Rust facets and direct-rustc planning #1037). The loop was: edit compiler, build (~2 min), 17 min SDK re-preparation, test. Every compiler edit paid it, including edits to src/oven/ that cannot reach stdlib compilation.
• stdlib-web is the heaviest component by a wide margin: its dependency closure is 53 packages against 20 for the default async,json,ordinal set, and it is last on the critical path.

[dannymeijer]

Comments and docstrings

A comment or docstring change should not invalidate the cache. The answer differs between the two halves of the hash.

Compiler source (Rust) — safe to exclude

A comment or rustdoc line in src/ or crates/ cannot change what output the compiler produces for a stdlib component. Rust doc comments become rustdoc for the compiler itself, which no component build observes. Stripping comments before hashing this half is sound.

It needs lexing rather than byte hashing, but proc-macro2 and syn are already workspace dependencies, so hashing a token stream is available without new surface. This composes with the exclusions already proposed — narrowing which files are hashed and normalising within a file are independent.

Incan source (.incn) — depends on what the digest gates

An Incan docstring does reach a component's output. emit_rustdoc_attrs in src/backend/ir/emit/decls/mod.rs emits source docstrings as #[doc = ...] attributes on generated public items, which prettyplease renders as /// lines; docstring is also carried on AST declarations (crates/incan_syntax/src/ast/decls.rs) and travels into the published surface alongside docstring_sections. Measured over the stdlib, doc comments are 29% of emitted Rust by volume.

The distinction that matters is what the digest is being used for:

• Gating the compiled artifact. A /// cannot change the resulting rlib. Stripping doc lines is sound, and it is what makes a doc-only edit a cache hit instead of a 17-minute rebuild.
• Gating published rustdoc or the manifest surface. There the docstring is the output, and stripping it would serve a stale artifact.

So this is one digest per consumer, not one digest with a global policy.

Hash input	Comments	Docstrings
compiler source (Rust)	strip	strip
.incn, gating the compiled artifact	strip	strip
.incn, gating published docs/manifest	strip	keep

Incan # comments are safe to strip in every case.

Correction

An earlier revision of this comment claimed stripping docstrings on the .incn side was a correctness bug outright. That is too strong: it holds only where the same digest also gates rustdoc or manifest publication. For the compiled artifact it is both safe and necessary — see the measured results in the following comment, where a docstring-only edit leaves the digest unchanged while a real semantic edit moves it.

[dannymeijer]

Measured: a compiled-effect digest, at HIR level, is ~550× cheaper than the thing it replaces

Step 1 in the description proposes narrowing the compiler-source hash by path. That is a heuristic — it needs a hand-maintained exclusion list, and it is wrong in both directions (a tests/ helper pulled into src/ silently starts mattering; a src/lsp/ edit still counts even though it cannot reach a component build). The snapshot suites already contain a mechanism that answers the question exactly instead of approximating it, and it is cheap enough to run on every command.

The mechanism

tests/stdlib_generated_rust_snapshot_tests.rs compiles real stdlib sources through run_on_compiler_stack + IrCodegen with set_sdk_provider_module_paths(...). No subprocess, no rustc, no SDK preparation. That is the interesting part, not the goldens: the compiler can be asked what it would produce for the stdlib, in-process, in seconds.

Run over the whole stdlib source set (104 .incn files), two variants:

Digest layer	Coverage	Cost	Output hashed
Emitted Rust	102 / 104	2.78 s	3.20 MB
HIR v0 + Body IR v0	103 / 104	1.80 s	3.19 MB

Against ~17 minutes of SDK preparation, the HIR variant is roughly 550× cheaper. The two files that fail are artifacts of compiling a file standalone rather than as part of its component project (hash/prelude.incn cannot resolve its sibling std.hash._hmac; toml.incn hits a real rust::toml_edit::Item::Value mismatch) — a production implementation runs per component project and would not see either.

HIR is the better layer, and it is already there

HirModule::render_snapshot() and BodyIrModule::render_snapshot() both exist and are already deterministic. Body IR is not a stub: it renders locals with ownership facts, places, operands, match arms with binding modes and last-use, and per-body runtime_requirements. Hashing those is strictly less work than emitting Rust, and it is the representation RFC 106 codegraph can actually reason about — an emitted-Rust digest answers "did the bytes change" and nothing else.

Comments and docstrings: the requirement is met, but only after normalization

Raw, both layers are span-sensitive, so an added line invalidates everything below it. Three spellings carry a byte offset, and the fix for each is presentational:

• span=N..N on declarations, locals, and params
• CompilerNodeId's #decl.N..N suffix
• CanonicalSymbolId::render_compact()'s trailing @N..N

Docstrings additionally reach Body IR as a leading expr const("...") statement, and reach generated Rust as /// lines — 29% of emitted stdlib Rust by volume (3.20 MB → 2.28 MB when stripped). Neither form can change a compiled rlib.

Normalizing those four, digest over the full stdlib:

Edit to io.incn	Normalized digest
baseline	23d5d563…
add a standalone # comment	23d5d563… — unchanged
add a docstring sentence	23d5d563… — unchanged
zero: u8 = 0 → zero: u8 = 1, same line count	e2fa225f… — changed

So the "a comment or docstring edit must not move the hash" requirement is satisfiable, and the negative control confirms the normalization is not merely erasing everything.

The equivalent at emitted-Rust level needs the same two undos and the tree already has both: canonical_projection::decoded_source_spellings() (which the snapshot suite added for exactly this reason — "a single added comment in io.incn rewrites every projected identifier below it") plus doc-line stripping.

Per-component invalidation works today

Bucketing by the same namespace-roots publication uses gives all ten component digests from one 1.86 s pass. Editing one file in io.incn:

stdlib-system   23c37a81…  →  10f253a8…

Every other component byte-identical — 10 of 11 buckets unchanged. Under the current key that same edit rebuilds all ten. Combined with publication_order(), this is step 2 and step 3 of the description with an exact trigger instead of a path guess.

The limitation, stated plainly

An HIR digest alone is not a sound cache key. It is upstream of emission, so an emitter-only compiler change is invisible to it and would produce a false hit. This is not hypothetical — it describes both fixes that just landed:

• bug - Rust enum struct variants cannot be pattern-matched from Incan in any spelling #1491's own commit message: "Lowering needed no change: it already produced Pattern::Struct for named…". The fix was typechecker + emitter.
• bug - comparing a list to an empty list literal emits an untyped vec![] #1493 was emitter-only (emit/expressions/mod.rs, calls.rs), and its effect was visible only as changed snapshots.

An HIR-level key would have served a stale SDK for both.

Recommendation

Two digests from one in-process pass, ~4.6 s worst case:

1. Emitted-Rust digest, decoded and doc-stripped, as the sound invalidation key — it sits downstream of every compiler stage that can change the artifact. Combined with the workspace lock and toolchain identity the current scheme already hashes, this replaces the compiler-source-tree hash outright and makes step 1's exclusion list unnecessary.
2. HIR + Body IR digest as the explanatory layer: per-component, graph-shaped, and the input RFC 106 needs to answer "which components does this compiler change actually reach" rather than "did anything change".

The first is what makes the cache correct. The second is what makes the answer legible — and it is the half that Jackhammer's deferring engine needs.

Verification

All figures above measured in-process against 0.6.0-dev.4 at 64b7fb2a6, over crates/incan_stdlib/stdlib (104 sources). Probes were temporary and are not committed; the tree is unmodified.

[dannymeijer]

Correction to the recommendation above

The measurements in the previous comment stand. The recommendation built on them does not, for two reasons.

The digest as scoped is unsound: it covers only the Incan half

The probe ran over crates/incan_stdlib/stdlib — 104 .incn sources — and the recommendation generalised from it. But a component's build closure also contains 24 files and 6,834 lines of pure Rust in crates/incan_stdlib/src: the runtime every generated component links against (frozen.rs, collections.rs, iter.rs, strings.rs, and the rest).

None of that is in the digest. Editing frozen.rs changes every component's behaviour and moves nothing. That is a false cache hit — strictly worse than the current scheme, which over-invalidates in the safe direction.

Any digest that gates SDK reuse has to span both languages. This is the "one graph across Incan and Rust" contract in RFC 106, applied to invalidation: one key, two producers.

Keying on emitted Rust points at the layer being removed

The recommendation named the emitted-Rust digest as the sound key and HIR as the explanatory layer. That inverts the right order once Incan connects directly at HIR rather than transpiling to Rust source.

The argument for preferring emitted Rust was the emitter blind spot — that #1491 (typechecker + emitter, "Lowering needed no change") and #1493 (emitter only) are invisible to an HIR digest. That blind spot is a property of the transitional pipeline, not a permanent limitation of HIR. Without an emitter there is no emitter blind spot, and HIR plus Body IR becomes complete by construction.

Building the invalidation engine on transpiler output means rebuilding it when that output goes away — the same reason this issue is dropping the path-exclusion list rather than maintaining one.

Corrected shape

One key, two language producers, both at body level:

Half	Producer	Status
Incan	HIR v0 + Body IR v0 render_snapshot(), normalised	exists; measured at 1.80 s for the whole stdlib
Rust	ra_ap_hir via rust_inspect	crate is in-tree at 0.0.325; item/signature level only today

The Rust half is the open work. extract_rust_item and rust_type_implements_trait reach signatures, not bodies, so a signature-level Rust digest would miss a body change in frozen.rs — the same blind spot criticised above, moved to the other language. It needs body-level reach through ra_ap_hir or MIR. A token-stream digest of the Rust source is an acceptable stopgap because it errs toward over-invalidation, which is safe.

The emitted-Rust digest keeps one job in the meantime: a transitional guard covering today's emitter, retired with it. Not the thing to design around.

[dannymeijer]

Progress, not a fix yet.

The site is identified. sdk_provider_store_identity (src/cli/commands/common.rs) folds hash_sdk_provider_compiler_source_tree over the whole compiler tree, excluding only .github .agents .git .incan benches docs examples target tests workspaces. Everything under src/ and crates/ therefore counts, so editing an inspection module, a CLI command, or the language server rebuilds all ten components. The doc comment explains why it is not the executable digest — test-only Rust changes rebuild the binary — but the cure is far wider than the disease.

The replacement exists and is measured. stdlib_effect_digest landed on 0.6.0-dev.4 in #1509: 2,284 ms over the real standard library against the ~17 minutes it is meant to replace, deterministic across runs. Three inputs, and the last two are not optional — the checked meaning of every .incn source, the Rust runtime under crates/incan_stdlib/src that every component links against, and transitionally the emitter's own sources, because while lowering to Rust still exists a change confined to emission alters output without moving any HIR.

What is deliberately not done. Nothing consumes the digest. Switching the cache key is the half where a mistake is a false hit and therefore a silently wrong build, so it is its own change. The review question for it is coverage, not cleverness: name an input that changes generated standard-library Rust and is not one of the three the digest folds.

This issue stays open until the key actually moves.

[dannymeijer]

Partial: #1520 landed the job-cap change and it is not the fix. Staying open, with the measurement and two corrections to this issue's own analysis.

What #1520 did. INCAN_TEST_CARGO_BUILD_JOBS was a flat ?= 2 regardless of host. Ten components are prepared serially, each compiling its own Rust dependency closure, so on an 18-core machine the whole preparation ran on about an eighth of the box. The cap now scales with the host (three quarters, floor of 2, explicit override still wins): 2 → 13 here.

Measured: 917s, against roughly 17 minutes before. About 10%, not the multiple the core count suggests — and that measurement is contaminated, because workspace cargo check runs were competing for the same cores. Either way the conclusion holds and it is the opposite of what the throttle implied: the time is not going where a parallelism cap would recover it. Take #1520 as cleanup, not a fix.

Step 1 is already done. hash_sdk_provider_compiler_source_tree excludes .agents, .git, .incan, benches, docs, examples, target, tests and workspaces, and the exclusion matches at any depth, so per-crate tests/ directories are covered too. The issue's "exclude tests/ explicitly" is satisfied.

Step 2 will not help a compiler-development loop, for now. Emission is an input to every component, so an emitter edit correctly invalidates all ten however the identities are cut. effect_digest.rs says as much about its own third fold — "The transitional half: remove this with the emitter." Per-component identity does help a stdlib-only edit, which is a real case, but it is not the one that costs the seventeen minutes. It becomes the right change once emission is no longer a global input.

Step 3 needs a measurement first. All ten components deliberately share one Cargo target directory to avoid recompiling shared dependencies ten times, and cargo holds a lock per target directory for the duration of a build. Running the levels concurrently against that shared directory would serialize on the lock; giving each component its own directory would parallelize but duplicate the dependency compilation, and stdlib-web alone closes over 53 packages. Which wins is an empirical question, not an obvious one.

Which leaves the actual cost: cargo compiling those dependency closures at all. That is what the direct-rustc work removes, and it is the only change here with a multiple in it rather than a percentage.