feat: context optimization — AST skeleton caching for Read tool token reduction

.gitignore

@@ -20,6 +20,9 @@ Thumbs.db
 .claude/
 .gstack/
 
+# vibecop context optimization cache
+.vibecop/
+
 # Environment
 .env
 .env.*

CHANGELOG.md

@@ -5,6 +5,30 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.4.1] - 2026-04-07
+
+### Added
+
+- **Context optimization** — intercepts Claude Code Read tool re-reads to reduce token consumption
+  - AST skeleton caching via `bun:sqlite` (WAL mode, zero-dep under bun)
+  - Smart-limits unchanged re-reads to 30 lines + skeleton injected via `additionalContext`
+  - `vibecop init --context` configures PreToolUse/PostToolUse/PostCompact hooks with conflict detection
+  - `vibecop context benchmark` shows projected savings for any project (runs under node, no bun required)
+  - `vibecop context stats` shows actual token savings after sessions (requires bun)
+- **MCP tool: `vibecop_context_benchmark`** — benchmark context optimization potential through the MCP server
+- **Shared AST utilities** — new `src/ast-utils.ts` with `findImports()`, `findFunctions()`, `findClasses()`, `findExports()`, `extractJsPackageName()`
+- Build target: `bun run build:context` produces `dist/context.js` (26KB, bun runtime)
+
+### Changed
+
+- Refactored 4 detectors (god-component, god-function, mixed-concerns, undeclared-import) to use shared AST utilities — reduced code duplication, same behavior
+- `god-function` removed 68 lines of local helper functions now shared via ast-utils
+
+### Internal
+
+- 42 new tests (610 total): ast-utils (28), skeleton (12), cache (15), integration (11), init --context (4)
+- `.vibecop/` added to `.gitignore` (SQLite cache is project-local state)
+
 ## [0.4.0] - 2026-04-06
 
 ### Added

CONTRIBUTING.md

@@ -22,21 +22,26 @@ src/
   cli.ts              # CLI entry point (commander)
   config.ts           # Config loading + Zod validation
   engine.ts           # File discovery, AST parsing, detector orchestration
+  ast-utils.ts        # Shared AST traversal (findImports, findFunctions, findClasses, findExports)
+  context.ts          # Context optimization entry point (hook handlers, requires bun)
+  context/
+    benchmark.ts       # Benchmark command (pure ast-grep, no bun:sqlite — runs under node)
+    cache.ts           # SQLite persistence (bun:sqlite, WAL mode)
+    session.ts         # File hashing, token estimation, extension checks
+    skeleton.ts        # AST skeleton extraction from source code
+    stats.ts           # Token savings reporting
   project.ts          # Package manifest parsing (dependencies, devDependencies)
   types.ts            # All shared types (Detector, Finding, DetectionContext, etc.)
+  init.ts             # Agent tool detection + config generation (vibecop init)
   detectors/
     index.ts           # builtinDetectors array — register new detectors here
-    empty-error-handler.ts
-    god-function.ts
-    sql-injection.ts
-    ...                # 22 detectors total
+    utils.ts           # Detector-specific helpers (makeFinding, test function detection)
+    ...                # 35 detectors across quality, security, correctness, testing
   formatters/
-    index.ts           # Formatter registry
-    text.ts            # Default human-readable output
-    json.ts            # JSON output
-    html.ts            # HTML report
-    sarif.ts           # SARIF for IDE/CI integration
-    github.ts          # GitHub Actions annotations
+    index.ts           # Formatter registry (text, json, html, sarif, github, agent, gcc)
+  mcp/
+    index.ts           # MCP server setup + tool registration
+    server.ts          # Tool handlers (scan, check, explain, context_benchmark)
   action/
     main.ts            # GitHub Action entry point
     diff.ts            # PR diff parsing
@@ -46,11 +51,10 @@ src/
 test/
   detectors/           # One test file per detector
   formatters/          # Formatter tests
+  context/             # Context optimization tests (skeleton, cache, integration)
+  mcp/                 # MCP server tests
   fixtures/            # Sample source files for testing
-  cli.test.ts
-  config.test.ts
-  engine.test.ts
-  project.test.ts
+  ast-utils.test.ts    # Shared AST utility tests
 ```
 
 ## How to Contribute
@@ -140,10 +144,11 @@ Add the detector to the README detector table with its id, name, category, and s
 Run all checks locally:
 
 ```bash
-bun run lint        # Biome linter — fix all errors
-bun run typecheck   # TypeScript strict — no type errors
-bun test            # All tests must pass
-bun run build       # Build must succeed
+bun run lint           # Biome linter — fix all errors
+bun run typecheck      # TypeScript strict — no type errors
+bun test               # All tests must pass
+bun run build          # CLI build must succeed
+bun run build:context  # Context optimization build must succeed
 ```
 
 ## Review Process

README.md

@@ -64,7 +64,26 @@ Agent writes code → vibecop hook fires → Findings? Agent fixes → Clean? Co
 }
 ```
 
-Three tools: `vibecop_scan`, `vibecop_check`, `vibecop_explain`.
+Four tools: `vibecop_scan`, `vibecop_check`, `vibecop_explain`, `vibecop_context_benchmark`.
+
+## Context Optimization
+
+Reduce token consumption by ~35% on Read tool re-reads. When Claude Code reads a file it's already seen, vibecop intercepts the Read and serves a compact AST skeleton instead of the full file. Unchanged files get smart-limited to 30 lines + skeleton context.
+
+**Requires bun runtime** (uses `bun:sqlite` for zero-dependency caching).
+
+```bash
+vibecop context benchmark  # See projected savings for your project
+vibecop init --context     # Configure hooks (Claude Code only)
+vibecop context stats      # View actual token savings after sessions
+```
+
+How it works:
+1. **First read** — full file passes through, skeleton is cached
+2. **Re-read (unchanged)** — smart-limited to 30 lines + skeleton injected via `additionalContext`
+3. **Re-read (changed)** — full file passes through with "file changed" note
+
+Skeletons include imports, function signatures, class outlines, and exports — enough for Claude to understand file structure without re-reading the full implementation.
 
 ## Benchmarks
 
@@ -114,7 +133,7 @@ Catches: god functions, N+1 queries, unsafe shell exec, SQL injection, hardcoded
 - [x] **Phase 2.5**: Agent integration (7 tools), 6 LLM/agent detectors, `vibecop init`
 - [x] **Phase 3**: Test quality detectors, custom YAML rules (28 → 35)
 - [x] **Phase 3.5**: MCP server with scan/check/explain tools
-- [ ] **Phase 4**: Context optimization (Read tool interception, AST skeleton caching)
+- [x] **Phase 4**: Context optimization (Read tool interception, AST skeleton caching)
 - [ ] **Phase 5**: VS Code extension, cross-file analysis
 
 ## Links

package.json

@@ -1,13 +1,14 @@
 {
   "name": "vibecop",
-  "version": "0.4.0",
+  "version": "0.4.1",
   "description": "AI code quality toolkit — deterministic linter for the AI coding era",
   "type": "module",
   "bin": {
     "vibecop": "./dist/cli.js"
   },
   "scripts": {
     "build": "bun build src/cli.ts --outdir dist --target node --external @ast-grep/napi --external @ast-grep/lang-python",
+    "build:context": "bun build src/context.ts --outdir dist --target bun --external @ast-grep/napi --external @ast-grep/lang-python",
     "build:action": "npx ncc build src/action/main.ts -o dist/action --source-map --license licenses.txt",
     "test": "bun test",
     "lint": "bunx biome check",
@@ -17,6 +18,7 @@
   },
   "files": [
     "dist/cli.js",
+    "dist/context.js",
     "README.md",
     "LICENSE"
   ],

product-docs/src/content/docs/agent-integration/claude-code.mdx

@@ -68,6 +68,54 @@ This scans only files with uncommitted changes, making the hook fast even in lar
 
 If `.claude/settings.json` already exists, `vibecop init` will skip it to avoid overwriting your configuration. In that case, add the `PostToolUse` hook manually by merging the JSON above into your existing settings.
 
+## Context Optimization (v0.4.1+)
+
+vibecop can intercept Read tool calls to reduce token consumption on re-reads. When Claude Code reads a file it has already seen in the current session, vibecop smart-limits the read to 30 lines and injects an AST skeleton (imports, function signatures, class outlines) via `additionalContext`.
+
+**Requires bun runtime** for SQLite caching.
+
+### Setup
+
+```bash
+npx vibecop init --context
+```
+
+This adds three hooks to `.claude/settings.json`:
+
+- **PreToolUse Read** -- checks if the file was read before, smart-limits re-reads
+- **PostToolUse Read** -- caches the AST skeleton after each read
+- **PostCompact** -- prunes old session data
+
+### Benchmark First
+
+Before enabling, see projected savings for your project:
+
+```bash
+npx vibecop context benchmark
+```
+
+This scans your codebase and shows per-file skeleton compression ratios and projected token savings. No bun required for the benchmark.
+
+### View Stats
+
+After working with context optimization enabled:
+
+```bash
+npx vibecop context stats
+```
+
+Shows total reads, cache hits, hit rate, and tokens saved across sessions.
+
+### How It Works
+
+1. **First read** -- full file passes through, skeleton is cached in `.vibecop/.vibecop-context.db`
+2. **Re-read (unchanged)** -- smart-limited to 30 lines + skeleton injected via `additionalContext`
+3. **Re-read (changed)** -- full file passes through with "file changed" note
+
+### Conflict Detection
+
+`vibecop init --context` checks for existing PreToolUse Read hooks before adding its own. If another tool already uses `updatedInput` on Read, vibecop skips to avoid conflicts (only one hook can modify `updatedInput` per tool call).
+
 ## Troubleshooting
 
 ### vibecop not found

product-docs/src/content/docs/agent-integration/mcp-server.mdx

@@ -3,7 +3,7 @@ title: MCP Server
 description: Use vibecop as an MCP server for Continue.dev, Amazon Q, Zed, and other MCP-compatible tools
 ---
 
-vibecop includes a built-in MCP (Model Context Protocol) server that exposes scan, check, and explain tools over stdio transport. This enables integration with any MCP-compatible AI coding tool.
+vibecop includes a built-in MCP (Model Context Protocol) server that exposes scan, check, explain, and context benchmark tools over stdio transport. This enables integration with any MCP-compatible AI coding tool.
 
 ## Starting the Server
 
@@ -30,7 +30,7 @@ Configure your MCP client to connect to vibecop:
 
 ## Available Tools
 
-The MCP server exposes three tools:
+The MCP server exposes four tools:
 
 ### vibecop_scan
 
@@ -72,6 +72,18 @@ Get detailed information about what a specific detector checks for.
 
 If the detector ID is not found, returns an error with the list of all available detector IDs.
 
+### vibecop_context_benchmark
+
+Benchmark context optimization potential for a project. Shows per-file skeleton compression ratios and projected token savings at different re-read rates. Does not require bun.
+
+**Parameters:**
+
+| Parameter | Type | Required | Default | Description |
+|-----------|------|:--------:|---------|-------------|
+| `path` | string | No | Current working directory | Directory to benchmark |
+
+**Response:** JSON object with `totalFiles`, `totalTokens`, `topFiles` (top 10 by token count with skeleton compression), `projections` (savings at 20%/40%/60% re-read rates), and `enableCommand`.
+
 ## Tool-Specific Setup
 
 ### Continue.dev
@@ -128,9 +140,10 @@ The MCP server is built on the `@modelcontextprotocol/sdk` package and uses stdi
 MCP Client (Continue.dev / Amazon Q / Zed)
   ↕ JSON-RPC over stdio
 vibecop MCP Server
-  → vibecop_scan  → engine.scan()
-  → vibecop_check → engine.checkFile()
-  → vibecop_explain → detector metadata lookup
+  → vibecop_scan              → engine.scan()
+  → vibecop_check             → engine.checkFile()
+  → vibecop_explain           → detector metadata lookup
+  → vibecop_context_benchmark → benchmark() (token savings projection)
 ```
 
 The server handles graceful shutdown on SIGINT and SIGTERM. All logging goes to stderr so it does not interfere with the MCP protocol on stdout.

product-docs/src/content/docs/architecture.mdx

@@ -8,10 +8,12 @@ description: How vibecop works internally — engine data flow, detector pattern
 ```
 vibecop CLI (Commander.js)
 ├── Scan Engine           — discovers files, loads AST, runs detectors, dedup by priority
-├── MCP Server            — stdio transport, 3 tools (scan, check, explain)
+├── MCP Server            — stdio transport, 4 tools (scan, check, explain, context_benchmark)
+├── Context Optimization  — Read tool interception, AST skeleton caching (bun:sqlite)
 ├── Init Wizard           — auto-detects AI tools, generates hook/rule configs
 ├── Custom Rules Engine   — loads .vibecop/rules/*.yaml, validates with Zod, runs via ast-grep
 ├── Config Loader (Zod)   — validates .vibecop.yml, merges defaults, per-rule config
+├── AST Utilities         — shared findImports, findFunctions, findClasses, findExports
 ├── Detectors (35)        — AST pattern matching via ast-grep (@ast-grep/napi)
 ├── Formatters (7)        — text, json, html, sarif, github, agent, gcc output
 ├── Project Analyzer      — parses package.json, requirements.txt, lockfiles
@@ -40,9 +42,17 @@ vibecop/
 │   │   ├── html.ts             — Single-file HTML report
 │   │   ├── agent.ts            — Token-efficient one-per-line for AI hooks
 │   │   └── gcc.ts              — GCC-style for editor integration
+│   ├── ast-utils.ts            — Shared AST: findImports, findFunctions, findClasses, findExports
+│   ├── context.ts              — Context optimization entry (hook handlers, requires bun)
+│   ├── context/
+│   │   ├── benchmark.ts        — Benchmark command (pure ast-grep, runs under node)
+│   │   ├── cache.ts            — SQLite persistence (bun:sqlite, WAL mode)
+│   │   ├── session.ts          — File hashing, token estimation
+│   │   ├── skeleton.ts         — AST skeleton extraction
+│   │   └── stats.ts            — Token savings reporting
 │   ├── mcp/
 │   │   ├── index.ts            — MCP module entry, server creation + transport
-│   │   └── server.ts           — Tool handlers: scan, check, explain
+│   │   └── server.ts           — Tool handlers: scan, check, explain, context_benchmark
 │   ├── detectors/
 │   │   ├── index.ts            — Detector registry (all 35 built-in detectors)
 │   │   ├── utils.ts            — makeFinding/makeLineFinding helpers
@@ -163,9 +173,36 @@ The engine handles these error conditions explicitly:
 MCP Client (Continue.dev / Amazon Q / Zed)
   ↕ JSON-RPC over stdio
 vibecop MCP Server (@modelcontextprotocol/sdk)
-  → vibecop_scan  → engine.scan()
-  → vibecop_check → engine.checkFile()
-  → vibecop_explain → detector metadata lookup
+  → vibecop_scan              → engine.scan()
+  → vibecop_check             → engine.checkFile()
+  → vibecop_explain           → detector metadata lookup
+  → vibecop_context_benchmark → benchmark() (project token savings projection)
 ```
 
 The MCP server reuses the same `scan()` and `checkFile()` functions as the CLI. Tool input schemas are defined with Zod. The server handles graceful shutdown on SIGINT/SIGTERM.
+
+## Context Optimization Architecture
+
+Context optimization intercepts Claude Code Read tool calls to reduce token consumption on re-reads. It has two build targets with different runtimes:
+
+| Build | Runtime | Contents |
+|-------|---------|----------|
+| `dist/cli.js` | node | CLI + MCP + benchmark (no SQLite) |
+| `dist/context.js` | bun | Hook handlers + stats (bun:sqlite for caching) |
+
+```
+Read tool call
+  ├── PreToolUse fires → bun dist/context.js --pre
+  │   ├── First read? → passthrough (+ skeleton if cached)
+  │   ├── Re-read, unchanged? → smart-limit to 30 lines + skeleton
+  │   └── Re-read, changed? → passthrough + "file changed" note
+  │
+  ├── Read tool executes (full or smart-limited)
+  │
+  └── PostToolUse fires → bun dist/context.js --post
+      ├── Record session read
+      ├── Extract AST skeleton (imports, functions, classes, exports)
+      └── Cache skeleton + token counts in SQLite
+```
+
+Skeletons are extracted using the same `@ast-grep/napi` parser as the detectors, via shared utilities in `src/ast-utils.ts`. The SQLite database uses WAL mode for concurrent access and is stored at `.vibecop/.vibecop-context.db` (gitignored).