[java] Add linux-x64 implementation of in process Copilot CLI

[edburns]

Supercedes #2295 .

This PR is the roll up of the agentic work done in the subtasks of #2166 . At each step of those subtasks, the CI was clean and all reviews were applied as appropriate.

PR 2295 — Reviewer's guide: In-process FFI runtime for the Java SDK

TL;DR

This PR does for the Java SDK what #1901 did for .NET and #1915 did for Rust: it adds an in-process connection mode that loads the Copilot runtime (runtime.node cdylib) as a native library via JNA, eliminating the need for a separate CLI child process. Currently scoped to linux-x64 only; the entire in-process API surface is marked @CopilotExperimental.

The PR also restructures the Java Maven project from a single module into a multi-module reactor to support publishing the native runtime binaries as separate classifier JARs alongside the existing SDK JAR.

---

What's in the native binary, where does it come from, and how is it loaded?

The binary: runtime.node

Despite the .node extension (a napi-rs naming convention), runtime.node is an ordinary platform-specific shared library (.so on Linux). It is a Rust cdylib produced by the src/runtime crate in github/copilot-agent-runtime. It exposes two front doors:

• napi front door — loaded by Node.js as a native addon (existing CLI path).
• C ABI front door — 5 extern "C" lifecycle/transport entry points callable by any language via FFI without Node.js.

The 5 C ABI entry points are:

Entry point	Purpose
copilot_runtime_host_start	Start the runtime host. Blocks up to ~30s while the worker boots. Returns a server handle (0 = failure).
copilot_runtime_host_shutdown	Shut down a runtime host by server handle.
copilot_runtime_connection_open	Open a bidirectional connection; registers an on_outbound callback for runtime→SDK data delivery.
copilot_runtime_connection_write	Write a JSON-RPC frame from the SDK into the runtime.
copilot_runtime_connection_close	Close a connection.

All JSON-RPC methods travel as data through this fixed 5-function transport; the export surface never changes as the method set grows.

Where it comes from (build-time)

The copilot-native Maven module's build fetches the binary from npm during generate-resources:

1. fetch-native.mjs reads the pinned version and SHA-512 integrity hash for @github/copilot-linux-x64 from nodejs/package-lock.json.
2. Runs npm pack to download the exact tarball, verifies it against the integrity hash.
3. Extracts runtime.node and the copilot CLI executable into a staging directory.
4. maven-jar-plugin packages them into a classifier JAR (copilot-sdk-java-runtime-<version>-linux-x64.jar) with the layout native/linux-x64/runtime.node.

How it's loaded (runtime)

1. PlatformDetector (303 lines) determines the classifier using os.name, os.arch, and on Linux, ELF PT_INTERP parsing to distinguish glibc vs musl — no subprocesses, no heuristics.
2. NativeRuntimeLoader (466 lines) resolves the binary in this order:
   • COPILOT_CLI_PATH env var → checks for runtime.node alongside the CLI.
   • Classpath resource native/<classifier>/runtime.node → extracts atomically to ~/.copilot/runtime-cache/<version>/<classifier>/runtime.node.
   • Falls back to runtime.node alongside the bundled copilot executable.
3. JnaNativeBinding (253 lines) loads the library by absolute path via JNA and maps each C ABI export. Enforces a one-library-per-process invariant (library handle is static, never unloaded). Duplicate loads from the same path are silently accepted; different paths are rejected.
4. FfiRuntimeHost (349 lines) orchestrates the lifecycle: starts the host, opens a connection, bridges the bidirectional JSON-RPC transport. The on_outbound callback (invoked by native threads) feeds received data into a QueueInputStream that the SDK's existing JsonRpcClient reads from.

---

Structural changes

Multi-module Maven reactor

The single-module java/pom.xml is now a parent POM (pom packaging) with two submodules:

Module	Artifact ID	Purpose
java/pom.xml	copilot-sdk-java-parent	Reactor parent. Not published to Maven Central (maven.deploy.skip=true). Holds the release profile (GPG signing) inherited by all submodules.
java/sdk/	copilot-sdk-java	The existing SDK JAR (~1.5 MB). All existing source moved here from java/src/ → java/sdk/src/.
java/copilot-native/	copilot-sdk-java-runtime	Native runtime module. Produces classifier JARs (currently linux-x64 only, ~20-26 MB).

Consumer dependency declaration

<dependencies>
    <!-- Pure-Java SDK (~1.5 MB) -->
    <dependency>
        <groupId>com.github</groupId>
        <artifactId>copilot-sdk-java</artifactId>
        <version>${copilot.version}</version>
    </dependency>
    <!-- Native runtime for linux-x64 (~20-26 MB) — needed only for in-process mode -->
    <dependency>
        <groupId>com.github</groupId>
        <artifactId>copilot-sdk-java-runtime</artifactId>
        <version>${copilot.version}</version>
        <classifier>linux-x64</classifier>
    </dependency>
</dependencies>

Consumer usage

CopilotClientOptions options = new CopilotClientOptions()
    .setConnection(RuntimeConnection.forInProcess());

CopilotClient client = new CopilotClient(options);
client.start().get();

---

New public API surface (all @CopilotExperimental)

Type	Description
RuntimeConnection (sealed class)	Base type for connection configuration. Factory methods: forStdio(), forTcp(), forUri(String), forInProcess().
StdioRuntimeConnection	Spawns a runtime child process, communicates over stdin/stdout (the default).
TcpRuntimeConnection	Spawns a runtime child process listening on a TCP socket.
UriRuntimeConnection	Connects to an already-running runtime at a URL.
InProcessRuntimeConnection	Loads the native library in-process — no child process spawned.
CopilotClientOptions.setConnection() / getConnection()	Entry point for selecting a connection type.

The RuntimeConnection API replaces the previous pattern of setting cliUrl, cliPath, useStdio, port, and tcpConnectionToken individually. When a RuntimeConnection is set, it takes precedence; conflicting legacy options cause IllegalArgumentException.

---

New internal packages

com.github.copilot.ffi (9 classes, ~1,752 lines)

Class	Lines	Role
FfiRuntimeHost	349	Lifecycle manager: start host → open connection → bridge I/O → shutdown.
JnaNativeBinding	253	JNA bindings for the 5 C ABI exports. Static library handle, one-per-process guard.
NativeBinding	131	Abstract contract for native operations (enables testing without real native library).
NativeRuntimeLoader	466	Locates runtime.node: env var → classpath → cache. Atomic extraction with file locking.
PlatformDetector	303	Determines platform classifier. ELF PT_INTERP parsing for glibc/musl detection on Linux.
QueueInputStream	119	Thread-safe bridge: native callback thread writes → SDK reader thread reads.
FfiOutputStream	63	Writes JSON-RPC frames from the SDK into the native runtime via connection_write.
OutboundCallback	46	JNA callback implementation for on_outbound.
ReaderThreadFactory	22	Named daemon thread factory for the reader executor.

Tests for FFI (6 files, ~2,054 lines)

Test class	What it covers
FfiRuntimeHostTest	Lifecycle, error handling, concurrent shutdown, callback drain.
JnaNativeBindingTest	Load guard, duplicate-path acceptance, different-path rejection, active callback tracking.
NativeRuntimeLoaderTest	Resolution order, atomic extraction, COPILOT_CLI_PATH override, cache reuse.
PlatformDetectorTest	All 8 platform classifiers, ELF parsing, edge cases.
QueueInputStreamTest	Thread-safe read/write, close semantics.
InProcessTransportIT	End-to-end integration test using the replay proxy with in-process transport.

---

CI/workflow changes

• New job java-sdk-inprocess in java-sdk-tests.yml: runs mvn clean verify -Pinprocess on ubuntu-latest (linux-x64). Uses continue-on-error: true while experimental.
• Path updates in existing jobs: java/target/ → java/sdk/target/ for surefire/failsafe reports and coverage data.
• JDK 17 cross-test: added -pl sdk to restrict to the SDK module (the native module requires JDK 25 build tools).
• Codegen workflows: adjusted working directories for the java/sdk/ module layout.

✅ Note that the existing java publishing jobs will continue to work as currently written.

---

Key design decisions (from ADR-007)

1. JNA over Panama FFM: JNA supports the Java 17 baseline with zero consumer configuration. Panama FFM requires Java 22+ and --enable-native-access flags. Performance difference is irrelevant (JSON-RPC I/O dominates).

2. Per-platform classifier JARs over monolithic JAR: A monolithic JAR with all 6 common platforms would be ~132 MB. Classifier JARs let consumers pull only their target platform (~20-26 MB each). An uber-JAR can be assembled via maven-assembly-plugin if needed.

3. Library-never-unloads pattern: The loaded native library is held in a static field and never released. Native worker threads outlive any individual FfiRuntimeHost instance; unloading would crash.

4. One library per process: Enforced by a process-wide guard, consistent with Rust, .NET, Go, and Python SDK implementations.

---

Diff statistics

• 107 commits, 1,582 files changed (mostly renames from java/src/ → java/sdk/src/)
• ~6,624 insertions, ~825 deletions
• New production code: ~2,117 lines (FFI + RuntimeConnection API)
• New test code: ~2,054 lines
• New build infrastructure: copilot-native/pom.xml (214 lines), fetch-native.mjs (114 lines)

Recommended review order

1. ADR-007: java/docs/adr/adr-007-native-bundling-strategy.md — context, options considered, decision rationale.
2. RuntimeConnection API: rpc/RuntimeConnection.java, rpc/InProcessRuntimeConnection.java, and rpc/CopilotClientOptions.java (the setConnection/getConnection methods).
3. FFI bridge (bottom-up): NativeBinding.java → JnaNativeBinding.java → FfiRuntimeHost.java → NativeRuntimeLoader.java → PlatformDetector.java.
4. Native module build: copilot-native/pom.xml and copilot-native/scripts/fetch-native.mjs.
5. Multi-module restructure: java/pom.xml (parent) and java/sdk/pom.xml (child).
6. CI: .github/workflows/java-sdk-tests.yml (new inprocess job, path updates).
7. Tests: ffi/ test package and e2e/InProcessTransportIT.java.

Implementation details.

Implemented agentically using https://aka.ms/coreai/shepherd-task/slides .

• Issues, including post-mortems: [Java] Embed Rust-based Copilot CLI Runtime and cease requiring Node.js #1917 .

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

[roji]

@edburns here are some comments from my review agent, hope these make sense. Happy to take another human look afterwards!

---

Requesting changes. The overall transport architecture broadly aligns with the other SDKs, and the large file count is mostly explainable: 1,522 of 1,596 files are byte-identical moves into java/sdk/. The module split is reasonable, but the published dependency graph, native ABI/lifecycle, and release validation still have blocking issues.

GitHub cannot attach inline review comments to unchanged files, so these relocation omissions are called out here:

• .github/workflows/java-publish-maven.yml:204,207 still references java/jbang-example.java, so release preparation will fail after the move to java/sdk/jbang-example.java.
• scripts/docs-validation/validate.ts:388-394 searches the parent POM for artifact copilot-sdk-java; it now falls back to 1.0.0-SNAPSHOT instead of validating the reactor's 1.0.11-preview.0-SNAPSHOT artifact.
• .github/actions/java-test-report/action.yml:7,11,15 still searches java/target/**; current CI logs report that no test reports were found even though results are under java/sdk/target/**.
• .github/workflows/java-smoke-test.yml:66,139 still points to the pre-move prompt path.

Please address the inline findings and these unchanged-file omissions before merging.

[roji]

Here are a few more comments.

Another thing I noticed is that while all other language SDKs automatically download the correct platform package with the native binary, the current approach in this PR requires users to manually take a dependency on e.g. the linux-x64 package, in addition to the platform-agnostic SDK package.

I don't know anything about how this kind of thing works with Java/Maven; is it impossible/not "the right way" to offer something that does this automatically (as all the other SDKs do)? Or maybe you're planning to look at that separately in a later PR (obviously completely fine too). Just raising the question.

[github-actions]

Java Codegen Fix — Automated Analysis

The java-codegen-fix workflow ran and found that the code generation itself succeeded — the regenerated output was already committed to this branch (commit 95a51d8).

Root cause of mvn verify failure

The build failure is not a code generation issue. It is a CI environment issue:

[ERROR] Rule 0: org.apache.maven.enforcer.rules.version.RequireJavaVersion failed with message:
[ERROR] JDK 25+ is required to build the Multi-Release JAR with the virtual-thread overlay.

This branch's java/sdk/pom.xml has an enforcer rule requiring JDK 25+, but the java-codegen-check workflow runner used a JDK version lower than 25.

What was regenerated

The workflow already committed the following changes (commit 95a51d8):

• SubagentCompletedEvent.java — added cancelled field
• FactoryAgentOptions.java — added reasoningEffort, contextTier, agent fields
• SandboxConfig.java — replaced gitAuth/ghAuth with new SandboxConfigAuth auth object
• SandboxConfigAuth.java — new generated type
• SessionFactoryApi.java — listRuns() now takes SessionFactoryListRunsParams
• SessionFactoryListRunsParams.java — added paging cursor fields
• SessionFactoryListRunsResult.java — added pagination metadata fields

What needs manual intervention

The java-codegen-check workflow needs to use JDK 25 to build this branch (which uses the Multi-Release JAR feature). The workflow configuration (.github/) is outside the scope of the automated fix. A human needs to ensure the CI runner for this branch is configured with JDK 25+.

Additionally, if any handwritten code in java/src/main/java/ or java/src/test/java/ needs to be updated to match the new generated types (e.g., callers of SandboxConfig that used gitAuth/ghAuth or callers of listRuns() that passed no arguments), those changes would also be needed — but they can only be validated once a JDK 25 environment is available.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• repo.maven.apache.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "repo.maven.apache.org"

See Network Configuration for more information.

Generated by Java Codegen Agentic Fix · sonnet46 86.5 AIC · ⌖ 4.25 AIC · ⊞ 18.8K · ◷

All comments addressed.

[edburns]

Here are a few more comments.

Another thing I noticed is that while all other language SDKs automatically download the correct platform package with the native binary, the current approach in this PR requires users to manually take a dependency on e.g. the linux-x64 package, in addition to the platform-agnostic SDK package.

I don't know anything about how this kind of thing works with Java/Maven; is it impossible/not "the right way" to offer something that does this automatically (as all the other SDKs do)? Or maybe you're planning to look at that separately in a later PR (obviously completely fine too). Just raising the question.

We followed prior art:

This is the same pattern used by DJL's PyTorch native artifacts (pytorch-native-cpu-2.5.1-linux-x86_64.jar, pytorch-native-cpu-2.5.1-osx-aarch64.jar, etc.), Netty's netty-tcnative-boringssl-static per-platform JARs, and others.

See the ADR.