fix(extract): resolve Python cross-file type references instead of ghosting them (#2363)

[Rishet11]

Closes #2363

What I found

The reported duplication is one defect with two id shapes, plus a second, separate defect that the issue title also names. Both are Python-only gaps against behavior other languages already have.

I first measured the reporter's layout with a single definition of PricePoint and four referring files: that comes out clean on v8 head, one node and four correct edges. The missing ingredient for the ghosts is a second same-named definition elsewhere in the corpus. With that, both reported shapes reproduce, and which one you get depends only on how many files refer to it:

ambiguous + 1 referring file
   pricepoint                                                     <- bare ghost

ambiguous + 4 referring files
   signal_intelligence_connections_prices_py_pricepoint           <- path-qualified ghosts
   signal_intelligence_fronts_agri_deviation_py_pricepoint
   signal_intelligence_fronts_upstream_watch_signals_py_pricepoint
   signal_intelligence_signal_resolution_py_pricepoint

Those are the four ids from the issue, character for character. The mechanism: each referring file mints its own sourceless stub with the bare id pricepoint; when there are several, they collide on that id but carry different origin_files, so _disambiguate_colliding_node_ids salts each into a path-qualified id before _rewire_unique_stub_nodes runs. So the "two different ID-generation schemes colliding" the issue describes is really one scheme observed at two corpus sizes.

1. Class-body field annotations were never collected. _python_collect_type_refs had exactly two call sites, function parameters and return types. Nothing walked a class_definition body, so the dataclass shape named in the issue title emitted no edge at all — a dropped edge, not a ghost:

@dataclass
class Quote:
    point: PricePoint    # before: no edge, no node

Java and TS both handle their equivalent today. Measured on identical corpora:

JAVA  b_quote_quote -> a_pricepoint_pricepoint  references field
TS    b_quote_quote -> a_base_pricepoint        references field
PY    (nothing)

2. An ambiguous class name leaves the reference stuck on a shadow stub. _rewire_unique_stub_nodes bails when the label is not globally unique, so with two modules defining PricePoint the sourceless stub survives and the edge stays on it — which is exactly the bare-lowercase ghost variant the reporter hit 7 times:

3 PricePoint nodes:
  pkg_a_base_pricepoint   src=base.py
  pkg_z_other_pricepoint  src=other.py
  pricepoint              src=''        <- ghost
  pkg_b_c_f -> pricepoint  references parameter_type

from pkg.a.base import PricePoint names the defining module exactly. Java resolves this with _resolve_java_type_references, PHP and C# with their own resolvers. Python had none — _collect_python_symbol_resolution_facts collects imports and calls, never type references. Java on the identical corpus gets it right:

2 PricePoint nodes, no ghost
  b_svc_svc_take -> a_pricepoint_pricepoint  references parameter_type

The fix

Both changes follow existing precedent rather than introducing anything new — no new edge type, no schema, exporter or affected changes.

• engine.py: a Python class-body walk emitting references with context field / generic_arg, mirroring the Java record-component branch, through the same _semantic_reference_edge emitter the parameter/return path already uses (it validates the context against REFERENCE_CONTEXTS).
• resolution.py + extract.py: _resolve_python_type_references, mirroring _resolve_java_type_references, keyed on the local binding so import X as Y resolves. Runs after _rewire_unique_stub_nodes, so it only handles the ambiguous remainder.

Two deliberate narrowings, both measured rather than assumed:

• The repoint set omits imports, which Java and C# both carry. A Python file-level import edge is resolved upstream and never lands on a bare-name stub — verified against ambiguous, aliased and unresolvable-module corpora — so including it would have been an untested no-op.
• The resolver early-returns unless shadow stubs exist, so its parse pass is only paid on a corpus that actually has unresolved references.

Before / after, real CLI

graphify extract . --code-only on one definition of PricePoint in pkg/a/base.py, a second in pkg/z/other.py, and a Quote dataclass importing the first:

Before — the field reference is absent entirely, and Quote is isolated:

E pkg_a_base -> pkg_a_base_pricepoint contains
E pkg_b_quote -> pkg_a_base_pricepoint imports
E pkg_b_quote -> pkg_b_quote_quote contains
E pkg_z_other -> pkg_z_other_pricepoint contains

After — the edge exists and binds to the imported definition, not the other same-named one:

E pkg_b_quote_quote -> pkg_a_base_pricepoint uses field

Verification

• 11 tests in tests/test_python_type_references.py. Checked out against base, 8 fail and 3 pass — the 3 are the pins and negative controls (the no-import bail-out, the plain x = 1 attribute, the already-working unambiguous case), which are supposed to be green both sides.
• One of them drives the real CLI and asserts on the written graph.json, since a unit test on extract() can pass while the shipped pipeline drops the result.
• The fix was staged in halves to confirm each test is load-bearing: the engine change alone flips the field tests, the resolver flips the ambiguity tests.
• Full suite: 4 failed / 3906 passed / 3 skipped. All 4 reproduce with the change reverted (test_ollama.py x3 and test_llm_backends.py, all reading real env vars), so the failure-set diff is empty.
• ruff clean. python -m tools.skillgen --check → check OK: 134 artifact(s).

Known limitations

• A reference whose name is ambiguous and which has no import naming its definition is still left on the stub. Guessing one of two equally-plausible classes would invent an edge; there is a test pinning this.
• Star imports (from pkg.a.base import *) are not used for disambiguation.
• Wide-corpus perf: the resolver re-parses Python files, like the Java and C# resolvers do. Gated on stubs existing, so a clean corpus pays nothing.

Relationship to #1231

#1231 targets this same area and has been conflicting since June. I did not build on it — it routes annotations through _SymbolUseFact, which is a larger redesign of which layer owns Python type references, and it emits context="annotation", which is not a member of REFERENCE_CONTEXTS. Happy to close this if you would rather revive that one.

[Copilot]

Pull request overview

This PR improves Graphify’s Python AST extraction so cross-file type annotations produce correct references edges and don’t leave behind “shadow” stub nodes when class names are ambiguous across modules (issue #2363). It brings Python closer to the existing Java/C#/PHP type-resolution behavior by collecting missing class-body field annotations and adding an import-aware post-pass resolver for remaining ambiguous stubs.

Changes:

• Emit references edges for Python class-body field annotations (including generic-arg contexts) during extraction.
• Add a Python type-reference resolver that repoints dangling edges from sourceless stubs to the imported definition when disambiguated by from ... import ... bindings.
• Add regression tests (including an end-to-end CLI run) to prevent both “dropped edge” and “ghost stub” regressions.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
tests/test_python_type_references.py	Adds regression coverage for class-body field annotations and ambiguous imported type resolution, including a CLI-level assertion on graph.json.
graphify/extractors/resolution.py	Introduces _resolve_python_type_references to repoint edges off sourceless stubs using import facts, then prune unreferenced stubs.
graphify/extractors/engine.py	Walks Python class_definition bodies to collect annotated assignments and emit references edges with field / generic_arg context.
graphify/extract.py	Integrates the new Python type-reference resolver into the extraction pipeline after _rewire_unique_stub_nodes and cross-file import resolution.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[graphify-labs]

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

Graphify reviewed this change.

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

---

Graphify review — findings

This PR adds Python cross-file type-reference resolution to the graphify extractor. It introduces a new _resolve_python_type_references pass (in resolution.py) that re-points dangling type-annotation edges from ambiguous sourceless "shadow" stub nodes onto the actual imported class definitions using each file's own imports, and drops any stubs left unreferenced; this pass is wired into the main extract flow for Python files. It also extends _extract_generic in engine.py to collect class-body field annotations (e.g. dataclass fields like point: PricePoint) and emit references/field edges for them. A new test file (tests/test_python_type_references.py) covers these behaviors, including field-annotation edges, ambiguous same-named classes across packages, aliased imports, and a CLI-based single-node-per-class check.

No blocking issues surfaced. 3 lower-confidence candidates did not survive cross-model review.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 1598 functions depend on the 496 functions this change touches.

Health — this change adds coupling hotspots:

• worse: extract() — 360 callers, 29 callees
• worse: _collect_python_symbol_resolution_facts() — 3 callers, 10 callees

Verification — 1598 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

• verification_scope: 1471 function(s) in the blast radius were not formally verified this run

· 2 more finding(s) on lines outside this diff (see the check run).

[Rishet11]

Self-review follow-up in c8b3780, correcting two things in what I first pushed.

The performance note in the PR body was wrong. I wrote that the resolver's parse pass is "only paid on a corpus that actually has unresolved references" and that "a clean corpus pays nothing". That is not true. Every stdlib or third-party annotation leaves a sourceless stub — this one-file corpus produces two:

import sqlite3
from pathlib import Path

def f(c: sqlite3.Connection, p: Path) -> None: ...

sourceless stubs: ['connection', 'path']

So ordinary code trips the gate and most real corpora do pay the pass, measured at ~15ms/file (1.18s across this repo's own 80 files under graphify/). Still bounded and still the same shape as the Java and C# resolvers, which re-parse unconditionally, but I should not have described it as rare.

I tried tightening the gate to "a stub label names a real definition in the corpus", which does shut it on the case above. I reverted it: the stub for from X import Y as Z carries the local alias Z, no definition is labeled Z, so the tighter gate silently disabled alias resolution. My own aliased-import test caught it. Both the real cost and the rejected alternative are now recorded in the docstring rather than left as a comment that reads better than the truth.

inherits was in the repoint set with no test covering it. Superclasses reach ensure_named_node from a different call site than type annotations, so none of the annotation cases exercised that path. Added test_ambiguous_superclass_resolves_through_the_import. It passes as written, so it pins existing behavior rather than fixing anything — but shipping an untested branch I had asserted was covered was the gap.

Now 10 tests. Suite 4 failed / 3905 passed, same four env-dependent failures that reproduce with the change reverted; ruff and skillgen --check clean.

One thing still open from the issue, which I do not want to paper over: I could not reproduce the path-qualified ghost variant (signal_intelligence_<path>_py_pricepoint) at all, so this PR does not explain where those four came from. My best guess was that per-file stubs collide and get salted apart by _disambiguate_colliding_node_ids before the rewire can collapse them, but every corpus I built that should trigger that came out clean, so I am not claiming it. @SatishRockzz's redacted graph.json would settle it. If it turns out to be a live third defect, it wants its own issue rather than being folded in here.

[graphify-labs]

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

Graphify reviewed this change.

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

---

Graphify review — findings

This PR adds Python cross-file type-reference resolution to the graphify extractor. In the engine, it extends _extract_generic to collect type references from class-body field annotations (e.g. dataclass fields like point: PricePoint), which previously were not walked. In resolution, it introduces a new _resolve_python_type_references function—wired into the extract pipeline after cross-file import resolution—that re-points dangling type-reference edges stuck on ambiguous sourceless "shadow" stubs to the correct imported definition using each file's own imports. The change also adds a new test file (tests/test_python_type_references.py) covering the field-annotation and ambiguous-name scenarios. The large set of listed changed symbols appears mostly incidental to the diff shown, which centers on extract.py, engine.py, and resolution.py.

No blocking issues surfaced. 5 lower-confidence candidates did not survive cross-model review.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 1600 functions depend on the 498 functions this change touches.

Health — this change adds coupling hotspots:

• worse: extract() — 361 callers, 29 callees
• worse: _collect_python_symbol_resolution_facts() — 3 callers, 10 callees

Verification — 1600 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

• verification_scope: 1473 function(s) in the blast radius were not formally verified this run

· 2 more finding(s) on lines outside this diff (see the check run).

[Rishet11]

Closing the loop on the one thing I said I could not explain. The path-qualified ghosts reproduce, and they share the bare stub's root cause — 271b130 adds the fixture, PR body updated.

The ingredient I was missing is a second same-named definition elsewhere in the corpus. With one, everything resolves cleanly, which is why my earlier repros came out empty. Add a vendor/legacy.py that also defines PricePoint, and the shape depends only on how many files refer to it:

ambiguous + 1 referring file
   pricepoint

ambiguous + 4 referring files
   signal_intelligence_connections_prices_py_pricepoint
   signal_intelligence_fronts_agri_deviation_py_pricepoint
   signal_intelligence_fronts_upstream_watch_signals_py_pricepoint
   signal_intelligence_signal_resolution_py_pricepoint

Those are @SatishRockzz's four ids character for character. Each referring file mints its own sourceless stub with the bare id pricepoint; with several they collide on that id but carry different origin_files, so _disambiguate_colliding_node_ids salts each into a path-qualified id before _rewire_unique_stub_nodes can collapse them. So the "two different ID-generation schemes colliding" is one scheme seen at two corpus sizes, and the 4-vs-7 split in the report is just which classes had multiple referrers.

That also predicts something checkable in the reporter's corpus: each of those 11 classes should have a second definition of the same name somewhere — a vendored copy, a tests/ fixture, a compat shim. The issue says each was confirmed defined once via grep -n "^class ", which would miss a second definition written as class PricePoint(Base): on an indented line, inside if TYPE_CHECKING:, or in a file that grep pass did not cover.

No behavior change from this commit — the resolver already handled the case, so it is coverage for a shape I previously could not assert. 11 tests now; 8 fail at base, the 3 that pass are the pins and negative controls. Suite 4 failed / 3906 passed, empty failure-set diff; ruff and skillgen clean. Also fixed the docstring wording Copilot flagged.

[graphify-labs]

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

Graphify reviewed this change.

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

---

Graphify review — findings

This pull request adds a new Python cross-file type-reference resolution pass to the extractor pipeline, targeting issue #2363 around duplicate/ghost nodes. It does two things: collects type references from class-body field annotations (the dataclass shape) in the generic extractor, and adds a new _resolve_python_type_references function in resolution.py that re-points dangling type-reference edges from sourceless "shadow" stubs onto the imported definition using each file's own imports. The changes wire this new resolver into extract(), add the function's import, and modify the class-body handling in engine.py to emit references edges for annotated fields. A new test file (tests/test_python_type_references.py) is added covering the field-annotation and ambiguous-name resolution scenarios. The large list of "changed symbols" appears to reflect renamed/reindexed rationale-comment identifiers across these files rather than substantive logic changes.

No blocking issues surfaced. 5 lower-confidence candidates did not survive cross-model review.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 1602 functions depend on the 500 functions this change touches.

Health — this change adds coupling hotspots:

• worse: extract() — 362 callers, 29 callees
• worse: _collect_python_symbol_resolution_facts() — 3 callers, 10 callees

Verification — 1602 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

• verification_scope: 1475 function(s) in the blast radius were not formally verified this run

· 2 more finding(s) on lines outside this diff (see the check run).