Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/workflows/claude-issue-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,6 @@ jobs:
uses: anthropics/claude-code-action@a017b830c03e23789b11fb69ed571ea61c12e45c # v1.0.30
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
track_progress: true
prompt: |
Analyze this issue for the cc-plugin-eval TypeScript evaluation framework.

Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/claude-pr-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ jobs:
- `npm run typecheck` - TypeScript errors
- `npm run lint` - ESLint issues
- `npm run format:check` - Formatting
- `npm run knip` - Dead code detection

### Step 2: Review the Diff
Run `gh pr diff ${{ github.event.pull_request.number }}` to see all changes.
Expand Down Expand Up @@ -120,4 +121,4 @@ jobs:
- **Suggestions**: General improvements (not covered by inline comments)

Be constructive. Focus on significant issues, not nitpicks.
claude_args: '--allowedTools "mcp__github_inline_comment__create_inline_comment,Bash(gh pr diff:*),Bash(gh pr view:*),Bash(gh pr comment:*),Bash(npm run typecheck),Bash(npm run lint),Bash(npm run format),Bash(npm run format:check),Bash(npm test),Bash(npm test:*),Read,Glob,Grep"'
claude_args: '--allowedTools "mcp__github_inline_comment__create_inline_comment,Bash(gh pr diff:*),Bash(gh pr view:*),Bash(gh pr comment:*),Bash(npm run typecheck),Bash(npm run lint),Bash(npm run format),Bash(npm run format:check),Bash(npm run knip),Bash(npm test),Bash(npm test:*),Read,Glob,Grep"'
10 changes: 5 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,7 @@ export async function fetchData(url: string) {

**Refactoring types**: Use `find_referencing_symbols` on a type from `src/types/` to find all usages before making changes.

**Tracing detection logic**: Detection is in `src/stages/4-evaluation/detection/`. The flow is `detectAllComponents` (in `core.ts`) → `detectFromCaptures` (in `capture-detection.ts`) → type-specific detectors (`agents.ts`, `commands.ts`, `hooks.ts`). Agent detection uses SubagentStart/SubagentStop hooks. Use `find_symbol` to navigate this chain.
**Tracing detection logic**: Detection is in `src/stages/4-evaluation/detection/`. The flow is `detectAllComponents` (in `orchestrator.ts`) → `detectFromCaptures` (in `capture-detection.ts`) → type-specific detectors (`agents.ts`, `commands.ts`, `hooks.ts`). Agent detection uses SubagentStart/SubagentStop hooks. Use `find_symbol` to navigate this chain.

**Adding a new component type**: Follow the type through all four stages using `find_referencing_symbols` on similar component types (e.g., trace how `hooks` is handled to understand where to add `mcp_servers`).

Expand Down Expand Up @@ -289,7 +289,7 @@ src/
│ └── detection/ # Decomposed detection logic
├── state/ # Resume capability, checkpointing
│ ├── index.ts # State exports
│ ├── core.ts # Core state operations
│ ├── operations.ts # State CRUD operations
│ ├── queries.ts # State query utilities
│ ├── updates.ts # State update operations
│ ├── display.ts # State display formatting
Expand All @@ -312,15 +312,15 @@ When adding support for a new plugin component type (e.g., a new kind of trigger
1. Define types in `src/types/`
2. Create analyzer in `src/stages/1-analysis/`
3. Create scenario generator in `src/stages/2-generation/`
4. Add detector in `src/stages/4-evaluation/detection/` (create new file, add to `core.ts`)
4. Add detector in `src/stages/4-evaluation/detection/` (create new file, add to \`orchestrator.ts\`)
5. Update `AnalysisOutput` in `src/types/state.ts`
6. Add to pipeline in `src/stages/{1,2,4}-*/index.ts`
7. Add state migration in `src/state/core.ts` (provide defaults for legacy state)
7. Add state migration in \`src/state/operations.ts\` (provide defaults for legacy state)
8. Add tests

### State Migration

When adding new component types, update `migrateState()` in `src/state/core.ts` to provide defaults (e.g., `hooks: legacyComponents.hooks ?? []`) so existing state files remain compatible.
When adding new component types, update `migrateState()` in \`src/state/operations.ts\` to provide defaults (e.g., `hooks: legacyComponents.hooks ?? []`) so existing state files remain compatible.

### Resume Handlers

Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ src/
│ └── 4-evaluation/ # Programmatic detection, LLM judge, metrics
│ └── detection/ # Decomposed detection logic
├── state/ # Resume capability, checkpointing
│ ├── core.ts # Core state operations
│ ├── operations.ts # State CRUD operations
│ ├── queries.ts # State query utilities
│ └── updates.ts # State update operations
├── types/ # TypeScript interfaces
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -476,7 +476,7 @@ src/
│ ├── aggregation/ # Result aggregation utilities
│ └── detection/ # Programmatic detection
│ ├── index.ts # Detection entry point
│ ├── core.ts # Core detection logic
│ ├── orchestrator.ts # Detection orchestration
│ ├── agents.ts # Agent detection
│ ├── commands.ts # Command detection
│ ├── hooks.ts # Hook detection
Expand All @@ -486,7 +486,7 @@ src/
│ └── types.ts
├── state/ # Resume capability
│ ├── index.ts # State exports
│ ├── core.ts # Core state operations
│ ├── operations.ts # State CRUD operations
│ ├── queries.ts # State query utilities
│ ├── updates.ts # State update operations
│ ├── display.ts # State display formatting
Expand Down
1 change: 1 addition & 0 deletions knip.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
"ignore": ["dist/**", "coverage/**", "node_modules/**"],
"ignoreDependencies": [],
"ignoreExportsUsedInFile": true,
"tags": ["-internal"],
"vitest": {
"entry": ["tests/**/*.test.ts", "tests/**/*.spec.ts", "vitest.config.ts"],
"project": ["tests/**/*.ts"]
Expand Down
1 change: 0 additions & 1 deletion src/config/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ export {
ExecutionConfigSchema,
EvaluationConfigSchema,
OutputConfigSchema,
type ValidatedEvalConfig,
} from "./schema.js";

export {
Expand Down
5 changes: 0 additions & 5 deletions src/config/schema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -314,8 +314,3 @@ export const EvalConfigSchema = z.object({
verbose: z.boolean().default(false),
max_concurrent: z.number().int().min(1).max(50).default(10),
});

/**
* Type inference from schema.
*/
export type ValidatedEvalConfig = z.infer<typeof EvalConfigSchema>;
5 changes: 4 additions & 1 deletion src/stages/4-evaluation/detection/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,10 @@
*/

// Core detection functions
export { detectAllComponents, detectAllComponentsWithHooks } from "./core.js";
export {
detectAllComponents,
detectAllComponentsWithHooks,
} from "./orchestrator.js";

// Capture-based detection
export {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/**
* Core Detection - Main detection orchestration.
* Detection orchestration - Main detection entry points.
*
* Provides the main entry points for component detection, combining
* all detection methods with proper priority ordering.
Expand Down
5 changes: 2 additions & 3 deletions src/state/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
*
* This module re-exports all state management functionality from focused submodules:
* - types: Shared type definitions (PipelineState, PipelineStage, etc.)
* - core: CRUD operations (create, save, load, paths)
* - operations: CRUD operations (create, save, load, paths)
* - updates: State update functions (updateStateAfter*, etc.)
* - queries: Query helpers (canResumeFrom, get*Scenarios, etc.)
* - display: Formatting and listing (formatState, listRuns)
Expand All @@ -23,7 +23,6 @@ export type {
CreateRunOptions,
PipelineStage,
PipelineState,
ResumeOptions,
} from "./types.js";

// Re-export core operations
Expand All @@ -34,7 +33,7 @@ export {
getStateFilePath,
loadState,
saveState,
} from "./core.js";
} from "./operations.js";

// Re-export update operations
export {
Expand Down
2 changes: 1 addition & 1 deletion src/state/core.ts → src/state/operations.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/**
* Core state management operations.
* State CRUD operations.
*
* Handles CRUD operations (create, save, load) and state file management.
* All functions in this module are pure or have clearly documented side effects.
Expand Down
8 changes: 0 additions & 8 deletions src/state/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -77,11 +77,3 @@ export interface CreateRunOptions {
config: EvalConfig;
runId?: string;
}

/**
* Options for resuming a pipeline run.
*/
export interface ResumeOptions {
runId: string;
fromStage?: PipelineStage;
}
Loading