Add high-level Project API for configuring analysis targets#2812
Open
oxisto wants to merge 10 commits into
Open
Add high-level Project API for configuring analysis targets#2812oxisto wants to merge 10 commits into
Project API for configuring analysis targets#2812oxisto wants to merge 10 commits into
Conversation
Introduces a new `de.fraunhofer.aisec.cpg.project` package as the primary entry point for analyses: `Project.from(path)` accepts a single source file or a whole repository and derives the lower-level `TranslationConfiguration` automatically. - `ComponentDefinition` replaces the parallel softwareComponents / topLevels maps with one object per component - `TargetEnvironment` describes the OS, architecture (incl. pointer width) and environment variables the project is assumed to run on; it is available to frontends via the translation configuration - A detection SPI split into `ComponentDetector` (invoked per directory during a central walk to find components) and `ProjectDetector` (invoked once for project-wide settings such as symbols, include paths or a compilation database); languages can implement either, standalone detectors such as `DirectoryComponentDetector` can be added manually - User configuration always takes precedence over detection; detection results are recorded on the `Project` for inspection `Path.topLevel` now prefers an include path over the component top-level if the file is not contained in the latter, so that translation units of dependencies (e.g., a standard library) get stable names relative to their include root. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`GoLanguage` detects one component per `go.mod` anywhere in the project tree (named after the module path) and derives the GOOS/GOARCH symbols from the target environment instead of implicitly assuming the host. The former `Project.buildProject` API moves into the opt-in `GoBuildDetector`, which uses an installed Go toolchain (`go list`) to resolve standard library and vendored dependencies; the go1.x build tags are now derived from the `go` directive in go.mod. `CLanguage` (and thus `CPPLanguage`) detects components and settings from a `compile_commands.json` in the project root or its `build` folder and wires the compilation database into the configuration. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The cpg_analyze and cpg_translate tools accept a new 'path' argument pointing to a source file or project directory. Project structure is auto-detected, e.g., components based on Go modules or a C/C++ compilation database. The analysis result reports the detected components and detection notes. This replaces the hand-built translation configuration in SetupConfig.kt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`AnalysisProject.temporary`, the `.codyze.kts` DSL and the console's analyze endpoint now build their translation configuration through `Project.from`, removing the duplicated language registration lists. As a side effect, they gain project auto-detection when no explicit sources, components or architecture modules are configured. cpg-neo4j keeps its explicit CLI option handling but reuses the shared `Project.defaultLanguages` list. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
oxisto
requested review from
KuechA,
fwendland,
konradweiss and
peckto
as code owners
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a new high-level de.fraunhofer.aisec.cpg.project.Project API to simplify configuring and running analyses, including auto-detection of components/settings (e.g., Go modules, C/C++ compilation databases) and a unified target environment model exposed to frontends via TranslationConfiguration.targetEnvironment.
Changes:
- Added
Project+ detection SPI (ComponentDetector/ProjectDetector) andTargetEnvironment, and wired the derived configuration intoTranslationConfiguration. - Updated multiple consumers (MCP, Codyze core/DSL, console, neo4j) to build configurations through
Project.from(...)and to share a centralized default-language list. - Added/updated tests for project detection (Go modules, C/C++ compilation database) and MCP project analysis output (components + detection notes).
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| cpg-neo4j/src/main/kotlin/de/fraunhofer/aisec/cpg_vis_neo4j/Application.kt | Reuses Project.defaultLanguages instead of duplicating optional language registration. |
| cpg-mcp/src/test/kotlin/de/fraunhofer/aisec/cpg/mcp/CpgAnalyzeProjectTest.kt | Adds MCP test coverage for analyzing a project directory with a compilation database. |
| cpg-mcp/src/main/kotlin/de/fraunhofer/aisec/cpg/mcp/SetupConfig.kt | Removes legacy translation configuration boilerplate (replaced by Project). |
| cpg-mcp/src/main/kotlin/de/fraunhofer/aisec/cpg/mcp/mcpserver/tools/utils/Serializable.kt | Extends MCP analysis result payload with components and detectionNotes. |
| cpg-mcp/src/main/kotlin/de/fraunhofer/aisec/cpg/mcp/mcpserver/tools/utils/Payloads.kt | Extends MCP analyze payload to accept filesystem path in addition to inline content. |
| cpg-mcp/src/main/kotlin/de/fraunhofer/aisec/cpg/mcp/mcpserver/tools/CpgAnalyzeTool.kt | Switches MCP analysis implementation to the new Project API and reports detection info. |
| cpg-mcp/build.gradle.kts | Adds optional test dependency on :cpg-language-cxx to enable compilation-db tests when available. |
| cpg-language-go/src/test/kotlin/de/fraunhofer/aisec/cpg/frontends/golang/GoProjectDetectionTest.kt | Adds unit tests for Go module component detection and env-derived GOOS/GOARCH. |
| cpg-language-go/src/main/kotlin/de/fraunhofer/aisec/cpg/frontends/golang/GoUtils.kt | Removes legacy Go-specific Project helper (replaced by new core Project + detectors). |
| cpg-language-go/src/main/kotlin/de/fraunhofer/aisec/cpg/frontends/golang/GoProjectDetection.kt | Adds lightweight Go project/module detection helpers and env symbol derivation. |
| cpg-language-go/src/main/kotlin/de/fraunhofer/aisec/cpg/frontends/golang/GoLanguage.kt | Implements detection interfaces on GoLanguage to participate in auto-detection. |
| cpg-language-go/src/main/kotlin/de/fraunhofer/aisec/cpg/frontends/golang/GoBuildDetector.kt | Adds opt-in tool-backed Go detector (go list / stdlib+vendor resolution). |
| cpg-language-go/src/integrationTest/kotlin/de/fraunhofer/aisec/cpg/frontends/golang/IntegrationTest.kt | Refactors integration test to use Project API + GoBuildDetector. |
| cpg-language-cxx/src/test/kotlin/de/fraunhofer/aisec/cpg/frontends/cxx/CXXProjectDetectionTest.kt | Adds unit test for C/C++ compilation database detection and symbol extraction. |
| cpg-language-cxx/src/main/kotlin/de/fraunhofer/aisec/cpg/frontends/cxx/CXXProjectDetection.kt | Adds C/C++ compilation database detection utilities for components/settings. |
| cpg-language-cxx/src/main/kotlin/de/fraunhofer/aisec/cpg/frontends/cxx/CLanguage.kt | Implements detection interfaces on CLanguage to participate in auto-detection. |
| cpg-core/src/test/kotlin/de/fraunhofer/aisec/cpg/project/ProjectTest.kt | Adds unit tests for new Project API behavior (single file, directory, detection precedence, disabling). |
| cpg-core/src/main/kotlin/de/fraunhofer/aisec/cpg/TranslationConfiguration.kt | Adds targetEnvironment to the core configuration and builder. |
| cpg-core/src/main/kotlin/de/fraunhofer/aisec/cpg/project/TargetEnvironment.kt | Introduces OS/architecture/env/sysroot model plus DSL builder. |
| cpg-core/src/main/kotlin/de/fraunhofer/aisec/cpg/project/ProjectDetector.kt | Introduces detection SPI types and a convention-based directory component detector. |
| cpg-core/src/main/kotlin/de/fraunhofer/aisec/cpg/project/Project.kt | Adds the high-level Project API and resolution into TranslationConfiguration. |
| cpg-core/src/main/kotlin/de/fraunhofer/aisec/cpg/graph/NodeBuilder.kt | Adjusts Path.topLevel selection order to prefer include paths when files lie outside component roots. |
| codyze-core/src/main/kotlin/de/fraunhofer/aisec/codyze/Project.kt | Refactors temporary project creation to use Project.from(...) and auto-detection. |
| codyze-core/src/main/kotlin/de/fraunhofer/aisec/codyze/dsl/Dsl.kt | Refactors script DSL project building to use Project.from(...) and pass configuration through translation {}. |
| codyze-console/src/main/kotlin/de/fraunhofer/aisec/codyze/console/ConsoleService.kt | Refactors console analysis endpoint to build configuration through Project.from(...). |
…ases Component detection now uses the same lookup as settings detection (compile_commands.json in the visited directory or its build folder), so that a database in <project>/build yields components rooted in the project directory instead of build/. This keeps translation unit names relative to the project sources. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 7, 2026
peckto
reviewed
Jul 8, 2026
| .optionalLanguage("de.fraunhofer.aisec.cpg.frontends.jvm.JVMLanguage") | ||
| .optionalLanguage("de.fraunhofer.aisec.cpg.frontends.ini.IniFileLanguage") | ||
| .also { builder -> | ||
| Project.defaultLanguages.forEach { builder.optionalLanguage(it) } |
Collaborator
There was a problem hiding this comment.
In the neo4j application, currently only the default languages are used from the Project.
Collaborator
|
How we want to handle multi-language projects? Should all source files from all languages be loaded, or just one? Some options currently work only with one language, like the |
Replace the boolean flags (`autoDetect`, `defaultPasses`) and flat
`registerLanguage()` / `component()` calls with three structured builder
blocks — `languages {}`, `passes {}`, `components {}` — that follow a
uniform auto-mode / explicit-override pattern:
- **Not calling the block** keeps auto-mode: all default languages,
the default pass pipeline, and language-based detectors.
- **Calling the block without `default()`** switches to explicit mode:
only what the user listed, detectors disabled.
- **Calling `default()` inside the block** combines both: defaults
plus any explicit additions.
The flat convenience helpers (`registerLanguage<T>()`, `component()`,
`detector()`) are kept as delegates to the new blocks for backward
compatibility.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Documents the auto-mode / explicit-override pattern for the
`languages {}`, `passes {}`, and `components {}` DSL blocks,
with code examples for each configuration scenario.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ctory
In auto-mode (no explicit `languages {}` block), instead of loading every
language available on the classpath, `ProjectBuilder` now scans the project
directory tree for file extensions and filters the candidate languages to
those that declare a matching extension in `Language.fileExtensions`.
Languages with an empty `fileExtensions` list are always included because
they use their own detection logic (e.g., `ComponentDetector`). Falls back
to loading all available languages when the path is not a directory or the
directory tree contains no files at all.
The same filtering applies when `languages { default() }` is used.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…terface Replace the two-interface design (ComponentDetector walked per-directory, ProjectDetector called once on root) with a single Detector that is called once on the project root and owns its own traversal. DetectionResult now carries both components and project-wide settings (symbols, include paths, compilation database) together. Language implementations (GoLanguage, CLanguage) implement Detector directly; standalone detectors (GoBuildDetector, DirectoryComponentDetector) follow the same contract. SKIPPED_DIRECTORIES is promoted to a package-level constant so every detector can share the same skip list. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove stale references to the removed ComponentDetector and ProjectDetector interfaces; replace with the single Detector interface throughout. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The API for configuring an analysis target has grown too complex:
TranslationConfigurationmixes ~28 constructor parameters covering what to analyze, language-specific parsing options, pass setup and debug flags, and every consumer (codyze CLI, codyze script DSL, console, MCP, neo4j) duplicated the same boilerplate — including four copies of the ten-lineoptionalLanguage(...)list.New
ProjectAPI (cpg-core)de.fraunhofer.aisec.cpg.projectis the new high-level entry point. The simplest usages are:with a builder DSL for more control (components, exclusion filters, target environment, and a
translation {}escape hatch to the underlyingTranslationConfiguration.Builder).ComponentDefinitionreplaces the parallelsoftwareComponents/topLevelsmaps.TargetEnvironmentdescribes the OS, architecture (incl. pointer width) and environment variables the project is assumed to run on. It defaults to the host, can be overridden for cross-target analysis, and is available to frontends viaTranslationConfiguration.targetEnvironment.Detection SPI, split into two concerns:
ComponentDetectoranswers "which units does this project consist of?" — invoked for every (non-excluded) directory during a central walk, so a detector only needs to recognize a single directory.ProjectDetectoranswers "which settings apply?" (symbols, include paths, compilation database) — invoked once on the project root.Languages can implement either interface; standalone detectors (e.g., the convention-based
DirectoryComponentDetector) can be added manually. User configuration always wins over detection, and detection results are recorded on theProjectfor inspection.Detectors
go.modin the tree, named after the module path;GOOS/GOARCHare derived from theTargetEnvironmentinstead of implicitly assuming the host. The formergolang.Project.buildProjectmoved into the opt-in, tool-backedGoBuildDetector(go list-based stdlib/vendor resolution); go1.x build tags now come from thegodirective.compile_commands.jsonin the root orbuild/folder, wiring the compilation database into the configuration.Consumers
cpg_analyze/cpg_translateaccept apathto a file or project directory with full auto-detection (incl. compilation databases); results report detected components and detection notes.SetupConfig.ktis gone.AnalysisProject.temporaryand the.codyze.ktsDSL build throughProject.from; both gain auto-detection when no explicit sources/components/modules are configured.Project.defaultLanguageslist.Notes
Path.topLevelnow prefers an include path over the component top-level when a file lies outside the component, so dependency translation units (e.g., Go stdlib) get stable, meaningful names (fmt/print.go) instead of collapsing to bare file names.targetEnvironmentin the C/C++ frontend yet (built-in macros, pointer widths); that is a planned follow-up, as is rebasingAnalysisProjectitself ontoProject.🤖 Generated with Claude Code