chore - make a module split free by keying identity on the exported path

[dannymeijer]

What

Reorganising a module today invalidates every consumer of the declarations that moved, even when the public API is byte-for-byte unchanged and every moved item is re-exported under its original path.

Splitting foo.incn into foo/mod.incn plus foo/detail.incn, with foo/mod.incn re-exporting everything, changes nothing a source-level consumer can see. It still renames every symbol that moved, still changes the unit's semantic digest, and still forces every dependent to recompile.

Why it happens

Two independent encodings of the declaring module path reach the final symbol, and both are ours:

1. RFC 120's incan-v1 projection encodes the complete CanonicalSymbolId, and encode_origin writes SymbolOrigin::Module(path) segments into the payload. Ordinary source functions emit as fn __incan_v1_<payload>; the codegen snapshot suite asserts this.
2. Emitted Rust mirrors Incan modules one-to-one (use crate::functions::col;, with the __INCAN_INSERT_MODS__ insertion point), so rustc mangles a def path we chose. There is no #[no_mangle] or #[export_name] in the emitter.

CanonicalSymbolId::origin also reaches the semantic digest through Body::canonical, so the unit's own identity moves too — not only its symbol.

Worth being precise about whose constraint this is: rustc imposes neither encoding. The genuinely external part is narrower — rustc has no "the dependency changed but my output need not", so any change to an upstream rlib recompiles downstream. That means holding symbol names stable while still producing a different rlib buys nothing. The property only pays if the split produces an identical unit, which means the Incan module structure must not reach the artifact at all.

What would fix it

Key identity on the exported path rather than the declaring path, at both layers:

• emit flat, with item names derived from the exported path;
• key SymbolOrigin on the exported path so the semantic digest does not move either.

A split preserving the public API then becomes invisible end to end: the upstream unit does not rebake, so no dependent ever sees a changed rlib.

Why this is not a small change

• Two declarations named foo in different submodules collide at a flat root and need a disambiguator that is not the declaring path.
• Backtraces, debug info, and incan inspect rust output all get less legible when the symbol stops naming where the code lives.
• A declaration exported under two paths, or none, needs a defined answer.
• SymbolOrigin is specified by RFC 120, which is closed. This is a superseding RFC, not a patch.

Evidence

Pinned by moving_a_declaration_into_a_submodule_changes_its_emitted_symbol in crates/incan_semantics_core/src/emitted_symbol.rs, and stated on encode_incan_symbol_identity so the property is met where it is created.

Recommendation

Not now. The cost is concentrated in naming and debuggability, and the benefit only materialises for projects that reorganise modules often. Nothing measured so far says that is common enough to pay for. Filed so the option is on record with its real price rather than rediscovered as a defect — a module split rebaking dependents is correct behaviour under the current model, not a bug.

[dannymeijer]

Measured cost

I filed this saying it was expensive without putting a number on it. Here is the number, measured on the stdlib corpus (1,252 declarations) with the harness in tests/declaration_identity_corpus.rs, extended to key identities with and without SymbolOrigin.

Keying	Distinct	Collision classes	Declarations affected
With origin (today)	1,252 / 1,252	0	0
Fully flat, no path at all	1,132 / 1,252	47	167 (13%)

Of the 47 classes, 34 are entirely private declarations and 13 involve at least one public one.

The public collisions are not accidents

They are the deliberate parallel-API pattern, and the signature discriminant cannot separate them because identical signatures are the entire point of a uniform interface:

• compress and decompress across bz2, deflate, gzip, lzma, raw, snappy, zlib, zstd — 8 modules, byte-identical signatures
• compress_stream / decompress_stream across 7 of the same
• encode, decode, decode_lenient, encode_stream, decode_stream across base32, base58, base64, base85, hex
• the read capability across clock, env, fs
• Duration as a model in both time and runtime

What that means for the proposal as filed

This proposal is exported-path keying, not full flattening, so the table's second row is the worst case rather than the estimate. The two halves separate cleanly:

Public declarations are free. An exported path keeps its exporting module segment — bz2.compress stays …codecs.bz2.compress — so none of the parallel-API collisions above arise. Splitting bz2.incn into bz2/mod.incn plus bz2/stream.incn with a re-export leaves every public key untouched. This is the case the proposal was written for and it works.

Private declarations are the entire cost. They have no exported path, so there is nothing to key them on but the declaring path — and keying them on the declaring path means the unit's own digest still moves on a split, which is the whole property we were buying. They are also where 34 of the 47 collision classes sit: _BYTE_TABLE across four base-N codecs, _NANOS_PER_SECOND across three time modules, and so on.

The only location-independent disambiguator left for them is content. That is more workable than it first sounds — two different private constants with the same name get different content, so different symbols, while two identical ones merge harmlessly, and any cascade from a content-derived name stays inside the unit being recompiled anyway. The price is that private symbols stop being readable in backtraces and incan inspect rust, and that naming becomes dependent on digesting, which constrains phase order in the emitter.

Code blast radius

• 187 references to SymbolOrigin:: across src/ and crates/
• 137 construction sites (module_declaration( / SymbolOrigin::Module()
• 1 of 196 codegen snapshots carries a projected symbol, so snapshot churn is negligible — I had assumed this was large and it is not
• plus a superseding RFC, since SymbolOrigin is specified by the closed RFC 120

Revised recommendation

Unchanged in conclusion, better founded. The public half is cheap and does what was wanted. The private half is the real work, and it trades readable private symbols for making module reorganisation free. Still not worth doing until module reorganisation is measured as frequent; but if it ever is, the public half can be done on its own for partial benefit, which was not obvious before measuring.

[dannymeijer]

Namespace-led subgraphs, not flattening at the root

Correcting the direction of this issue. Flattening at the root was the wrong shape, and the measurement above is the argument against it rather than for it: the 13% collision rate is what full flattening costs, and the public half of it is unavoidable by construction.

The parallel-API pattern is the proof. compress and decompress across eight codec modules, encode/decode/decode_lenient/encode_stream/decode_stream across five base-N modules, the read capability across clock, env and fs — every one of those has a byte-identical signature in every module, because a uniform interface is the entire point. No signature discriminant can separate them. A flat root namespace has nothing left to disambiguate with.

Scope identity to the enclosing namespace and the problem dissolves. Every one of the 47 collision classes is cross-namespace. Within a single namespace, two declarations sharing a name and signature is a duplicate definition — visible to the author at the point of writing, and fixable, unlike a cross-namespace clash which is neither.

The boundary already exists in the tree

src/frontend/library_exports.rs:1680 already treats a leading _ (but not __) as not exported, and the stdlib already relies on it:

hash/_core.incn      hash/_hmac.incn        hash/_streaming.incn
encoding/_shared.incn                       compression/_auto.incn
compression/_core.incn                      regex/_core.incn   regex/_replacement.incn

Those are internal details of std.hash, std.encoding, std.compression and std.regex. Everything needed to draw a subgraph boundary is present today.

What is missing is that nothing keys on it. canonicalize_source_module_segments (src/frontend/module.rs:342) collapses mod, __init__ and prelude, and knows nothing about _. So hash/_core.incn becomes the namespace std.hash._core rather than a detail inside std.hash, and a declaration moving between _core.incn and _streaming.incn changes namespace when it should not have left one.

This is also why the two collisions worth looking at are already benign under the right scoping: encoding/_shared.incn::_ASCII_PRINTABLE against encoding/bech32.incn::_ASCII_PRINTABLE is a detail of std.encoding against a declaration in std.encoding.bech32 — different subgraphs, no competition.

What this is not

It is not flattening the path. hash._core.foo must not become hash.foo; that would collide with a real hash.foo and would change identities that are correct today. A declaration keeps its location. What changes is that its reorganisation-stable scope becomes the nearest enclosing exported namespace, and that identity within a subgraph needs only local uniqueness.

Where it meets RFC 106

RFC 106 already requires a contains edge kind, and says a node identity "should include enough logical information to distinguish package, module path, stable declaration anchor, node kind, and relevant source identity". The graph already has containment. What it lacks is a namespace boundary distinct from the file tree — containment currently mirrors the directory layout rather than the import surface.

So this is a graph-shape change plus a scoping rule, landing at a named function, rather than the symbol-model rewrite this issue originally proposed. It also no longer needs private declarations to carry content-derived names, which was the ugliest part of the earlier sketch.

What is still open

Whether the _ convention should remain the only way to declare a module internal, or whether module visibility should be explicit. The convention is already load-bearing in library_exports.rs, so the graph can be built on it now; making it explicit is a separate language question and does not block this.