feat(eval): cover unresolved-symbol fallback in pre-edit gate

CHANGELOG.md

@@ -10,7 +10,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Added
 
 - **Context diagnostics**: `codebase_context` now supports an optional `diagnostic` flag across MCP, OpenCode, and Pi. It exposes bounded routing, retrieval, and evidence-pack traces for troubleshooting without changing normal text output.
-- **Pre-edit evaluation gate**: `codebase_edit_context` now has a documented CI evaluation gate. The budget gate supports a `minGraphNeighborRecall` threshold, and `npm run eval:pre-edit:ci` runs the deterministic mock-embedding baseline (Hit@5 1.0, MRR@10 1.0, graph-neighbor recall 1.0) with thresholds in `benchmarks/budgets/pre-edit.json`.
+- **Pre-edit evaluation gate**: `codebase_edit_context` now has a documented CI evaluation gate. The budget gate supports a `minGraphNeighborRecall` threshold, and `npm run eval:pre-edit:ci` runs the deterministic mock-embedding baseline (Hit@5 1.0, MRR@10 1.0, graph-neighbor recall 1.0) with thresholds in `benchmarks/budgets/pre-edit.json`. The golden dataset covers a resolvable caller lookup, a target-only lookup, and an unresolved-symbol fallback that must route to the conceptual search pack.
 
 ### Fixed
 

benchmarks/golden/pre-edit-context.json

@@ -39,6 +39,22 @@
         "filePath": "src/eval/runner.ts",
         "symbol": "runEvaluation"
       }
+    },
+    {
+      "id": "edit-context-unresolved-fallback",
+      "query": "how is context evidence about a code target assembled for an agent before editing",
+      "queryType": "definition",
+      "retrievalMode": "edit-context",
+      "args": {
+        "symbol": "resolveDefinitelyNonExistentSymbol",
+        "tokenBudget": 1200
+      },
+      "expected": {
+        "filePath": "src/tools/edit-context.ts",
+        "symbol": "formatResolutionRisk",
+        "expectedRoute": "search",
+        "expectedOutcome": "results"
+      }
     }
   ]
 }

docs/pre-edit-context-design.md

@@ -44,21 +44,21 @@ The acceptance gate is no regression in the existing `agent-context` and represe
 
 ## Baseline (mock embeddings, 2026-08-06)
 
-The gate runs in CI through `npm run eval:pre-edit:ci` with the deterministic mock-embedding provider and `benchmarks/budgets/pre-edit.json`.
+The gate runs in CI through `npm run eval:pre-edit:ci` with the deterministic mock-embedding provider and `benchmarks/budgets/pre-edit.json`. The dataset (`benchmarks/golden/pre-edit-context.json`) covers three cases: a resolvable caller lookup, a resolvable target-only lookup, and an unresolved-symbol fallback that must route to the conceptual `search` pack and surface the expected implementation chunk.
 
 | Metric | Baseline | Budget threshold |
 |---|---|---|
 | Hit@5 | 1.0000 | `minHitAt5` 1.0 |
 | MRR@10 | 1.0000 | `minMrrAt10` 1.0 |
 | Graph-neighbor recall | 1.0000 | `minGraphNeighborRecall` 1.0 |
-| p95 latency | 687 ms | `p95LatencyMaxAbsoluteMs` 5000 |
-| Response tokens average | 358.5 | `maxContextResponseTokensAverage` 600 |
-| Response tokens p95 | 399.5 | `maxContextResponseTokensP95` 800 |
+| p95 latency | 672 ms | `p95LatencyMaxAbsoluteMs` 5000 |
+| Response tokens average | 314.3 | `maxContextResponseTokensAverage` 600 |
+| Response tokens p95 | 394.9 | `maxContextResponseTokensP95` 800 |
 | Response tokens max | 404 | `maxContextResponseTokensMax` 1000 |
 | Duplicate candidate ratio | 0.0 | `maxContextDuplicateCandidateRatio` 0.5 |
-| Selected-file ratio | 0.7857 | `minContextSelectedFileRatio` 0.5 |
-| Hit@5 per 1k response tokens | 2.789 | `minContextHitAt5Per1kResponseTokens` 1.0 |
-| MRR@10 per 1k response tokens | 2.789 | `minContextMrrAt10Per1kResponseTokens` 1.0 |
+| Selected-file ratio | 0.8571 | `minContextSelectedFileRatio` 0.5 |
+| Hit@5 per 1k response tokens | 3.181 | `minContextHitAt5Per1kResponseTokens` 1.0 |
+| MRR@10 per 1k response tokens | 3.181 | `minContextMrrAt10Per1kResponseTokens` 1.0 |
 
 The budget gate supports the `minGraphNeighborRecall` threshold for datasets whose queries assert graph neighbors; datasets without graph-neighbor expectations are unaffected because the metric is only compared when present.