Repository navigation
connectorsmcp: add the Dex Connectors MCP server, its CI and its release - #657
Merged
Merged
Conversation
Add AgentToolMetadata with Effect(), the sealed AgentTool interface, NewQueryAgentTool and NewMutationAgentTool, and RunAttempt, which decodes arguments strictly, runs exactly one attempt through RunQuery or RunMutation, and classifies it. Classification follows the manifest's agent.branches listing first; an unlisted Mutation branch is unknown and an unlisted Query branch is rejected. The SDK only tightens it: uncertain and a defect that the SDK selected after Invoke on a Mutation are unknown. HasInvokedOperation reports whether Invoke ran, and panics are recovered into ErrAgentToolPanicked. Add projectconfig.LoadDexWebConnectionsFile to load a Dex Web connections.json by path; LoadFromEnvironment now calls it. RunQuery and RunMutation keep their behavior. The change is additive and needs a minor release (v0.35.0).
Treat a Mutation that uses the singledispatch Fence as unknown on any branch other than the happy path when the Fence's checkpoint cannot prove that no request reached the provider: a claimed checkpoint in this attempt without canResend, or a prior checkpoint that is unreadable, foreign, invalid, or does not allow this attempt to send. RunAttempt observes the Fence through a wrapper of the Step context. Document that other Mutations rely on their agent.branches listing. Report argument decoding and result encoding failures by category or Go type only, so custom Unmarshaler and Marshaler text never reaches the error, and add ErrAgentToolResultUnencodable. Require ToolName to equal the connector ID, an underscore, and the operation ID in connectorctl's snake case, with shared test vectors in sdkgo/testdata/agent_tool_names.json that connectorctl also checks.
…tcomes Remove the Dex context wrapper from Mutation agent tool attempts. Dex shows a recorded heartbeat only to the next attempt, so the singledispatch Fence now reports its first decision and its claimed and canResend checkpoints, by Call ID and attempt, to an internal fenceobservation registry that RunAttempt observes. The operation receives the caller's own Dex context, and heartbeats the operation records itself cannot change the result. Align classification with Dex agents: uncertain and a Mutation defect are always unknown, and an unlisted Query branch, including defect, is the new outcome failed. The Fence rule still tightens other non-happy Mutation branches to unknown. Make manifest validation reject agent.branches uncertain: rejected, and document that hosts ignore a Mutation defect listing.
Make the agent tool fence observation monotonic: once a Fence reports a claim or withholds a send, nothing clears the possible dispatch, not even canResend. Concurrent attempts with the same Call ID, connection, and attempt share one observation, so a canResend in one of them can no longer turn another's possibly delivered claim into rejected; sharing can only make results more conservative. Key observations by the connection as well, so calls on other connections stay isolated. An attempt that sent, saw canResend, and then selected a rejected branch is now unknown.
An operation can rewrite the Connection of the Call it receives before it constructs a Fence, which made the Fence report its claim under a key RunAttempt never looked up. Key observations by Call ID and attempt only. Because a possible dispatch never clears, attempts that share a Call ID and attempt on other connections can only become more conservative.
Add the connectorsmcp Go module with the dex-connectors-mcp/call-state/v1 store (per-key in-process mutex plus flock with inode re-verification, durable temp-file writes with file and directory sync, FORMAT check, retention pruning under the key lock), canonical JSON helpers, and an internal standalone dex.Context that mirrors dex.NewStandaloneContext until Dex Go SDK 1.41.0 is released.
…tcomes Add the tool set builder (selection, naming decision J, Dex convention and connector identity _meta), the tools/call executor (reads with inline retry, keyed writes under the call-state protocol, outcome mapping), the stateless Streamable HTTP endpoint with Host and Origin checks, hot reload, and a fake notes connector with a counting provider for tests.
…security Vendor the Dex MCP conventions fixture from superdurable/dex main 18a5b6a90 and check a reference classifier against every case, then classify the server's tool definitions and answers with it. Cover tool naming, the connectionsConfigDigest vectors, the mirrored Call ID, hot reload with renames, the Host and Origin matrix, and both MCP protocol eras through the go-sdk client.
Helper server processes re-executed from the test binary send themselves SIGKILL before and after the first record write, after the attempt, and after the result is stored, or are killed by the test while the provider holds the request. Re-issuing the key in a new process never sends a non-idempotent write twice and applies an idempotent write once. Also cover same-key calls across goroutines and processes, untrustworthy records, and a read-only state directory.
serve validates a loopback listen address, resolves the connections file (flag, DEX_CONNECTOR_CONFIG_FILE, the $HOME default, then hosted), takes the server.lock instance lock (exit 3 names the running URL), writes server.json before printing the JSON ready line, rewrites it on reload, and drains on SIGINT, SIGTERM, or stdin end of file. The release workflow replaces cmd/dex-connectors-mcp/zz_generated_registry.go; on main it links no connector.
Add connectorsmcp/README.md with the commands, flags, connections file resolution, selection and connectionsConfigDigest, tool naming, the ready line and server.json schema, tool _meta keys, the outcome mapping, inline retry, the call-state v1 format and protocol, and security. Add the server to docs/architecture.md, and test the internal attempt context.
Trim internal comment blocks to the reason or invariant; the README holds the contract. Rename argument and digest helpers away from normalize. Assert that an unknown tool answers -32602 with HTTP 400 under MCP 2026-07-28, which the MCP Python SDK 2.3 reads as the JSON-RPC error with its rejected outcome.
…the retry window Address review findings: - An existing key answers only the call that first used it: connector, operation, connection, project, and arguments must match. Another call is rejected with its state untouched. A vanished tool name matches only a name the record's tool had; an unknowable name answers unknown. - Records keep a monotonic possiblyApplied flag, set by an unobserved or retried attempt that may have reached the provider. While it is set no outcome is rejected. - The retry window is checked again after each retry delay, and every attempt's deadline stops at the window end. - tools/call may carry expectedConnector, expectedConnection, and expectedOperation; a mismatch with the record, or with the tool's binding when the key has no state, is refused with outcome rejected. - Empty tool lists in --connection and --connections-config are usage errors, and a config file with data after its document is refused.
Cover empty tool lists and trailing config data as exit-code-2 usage errors naming the entry. Document full key identity, the expected-identity guard, possiblyApplied, and the retry-window deadline in the README, and write U+2028 and U+2029 as escapes instead of raw characters.
Kill the server after an idempotent attempt that applied but asked to retry, before possiblyApplied is stored; the resumed call must answer unknown, not the later listed rejection. Also compare the record bytes around a refused key reuse.
Add the release-connectors-mcp job to the Release workflow. Each run links every connector whose latest complete Go release requires sdkgo v0.35.0, carries the generated agent tools, and exposes an operation. A hold rule keeps the largest set that builds and passes tests: a failing upgrade stays at its previous version, a failing new connector is left out, and a previously linked connector is never dropped or downgraded silently. The job commits the registry and go.mod on a commit off main, tags it connectorsmcp/vX.Y.Z, publishes darwin and linux archives with checksums.txt and a lock, and uploads connectors-mcp-release.complete last. Its failure never blocks connector releases or the catalog. The catalog job publishes connectors-mcp/index.json for dexcli from the complete releases. CI tests connectorsmcp with Go 1.25 against the repository's sdkgo when connectorsmcp/ or sdkgo/ changes.
- Create the off-main release commit with --allow-empty, so a server-only change or the last withdrawal still ships. - Never block the catalog on the index: a failed rebuild republishes the deployed index or deploys without one, with a warning. - Index only archives whose GitHub digest or downloaded bytes match the lock; a failing older release is left out, a failing newest one fails the rebuild. - Link a connector only when its go.mod declares this repository's module for its directory. - Release an empty server when the last linked connector is withdrawn, so the index stops offering it.
Records store connectionTarget, a SHA-256 of the connection's non-secret target fields (connector, connection, provider, auth method, shared connection, and canonical settings), never credentials, their expiry, or the module path. Before every attempt of an existing key, including after a retry delay, the current binding must have the same target. Otherwise no attempt runs and the key settles: rejected only with notSent proof and nothing possibly applied, else unknown. A credential refresh keeps the target.
Describe connectionTarget, which rebinding cases settle a running key, the cases Dex Web's connections.json cannot reveal (same settings with another account, multi-method selection), and the account identity Dex Web and projectconfig would need to expose to detect them.
…health An older release whose download fails now keeps its deployed index entry when that entry still matches the release's assets; only a mismatched or deleted asset drops it. The index step records its outcome (ok, fell_back, omitted, dropped_versions), the catalog still deploys, and a check-connectors-mcp-index job after the deploy fails on a degraded outcome. Document the repair path.
…r its lock A keyed call pinned the tool set at request entry, so a request that waited on the key's lock while the connection was rebound could resume the key with the old binding. After taking the lock the call now takes the current tool set and resolves its tool, guard, record identity, and target check from it; a new key records the target of the same snapshot its first attempt uses, and each inline retry still re-resolves. Add concurrency tests for a rebind while a second request waits on the lock and between request entry and a new key's first attempt.
…ndow Explain that a keyed call resolves its binding from the tool set current after it holds the key's lock and before every attempt, and that a Dex Web rebind landing during an attempt's own execution is not closed by the server: the planned sdkgo projectconfig guard (#655) resolves credentials and the connection target, including the upcoming authorizationId, from one snapshot.
A deployed entry reused without a GitHub digest or lock to confirm its SHA-256 now yields reused_unverified, and an index that lists nothing while a complete release exists yields empty. check-connectors-mcp-index fails on both; an empty index stays ok before the first release.
Reads kept the binding from request entry for every attempt, so a rebind during the wait or a retry delay could pair old settings with the new credentials. Each read attempt, including the read that runs when call state is unavailable, now resolves its tool from the current tool set and runs it from that snapshot; a tool that disappeared or changed target between attempts stops the read with rejected. Correct the README and the ToolSet comment that still described pinned tool sets, and extend the residual-window paragraph to reads.
# Conflicts: # sdkgo/projectconfig/README.md
sdkgo v0.35.0 is tagged, so the module now builds with GOWORK=off. Dex Go SDK 1.41.0 is not released yet; the internal attempt context stays.
# Conflicts: # Makefile
…rivately tools/call results bypass the go-sdk tool dispatch that sets resultType, so results under MCP 2026-07-28 lacked the field that version requires and the Python mcp 2.3 client refused them. Every tools/call result, including error, replayed, and truncated ones, now carries resultType complete when the session's protocol is 2026-07-28 or later, as go-sdk decides it, and older protocol versions stay without it. Listings now carry cacheScope private, since the tool list depends on the user's connections, and a JSON-RPC rejection logs outcome rejected with the reason unknownTool or expectedIdentityMismatch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the Dex Connectors MCP server (
dex-connectors-mcp, Go moduleconnectorsmcp/), which offers Connector Query and Mutation operations whose manifest setsagent.exposed: trueas MCP tools to any agent harness, plus its CI and release pipeline. New connectors become available through a connectors release, without a dexcli release. Built on sdkgo v0.35.0 (sdkgo.AgentTool, #651; cross-process refresh lock, #648) and the generated Go agent tools (#656).Server (
connectorsmcp/)/mcp, loopback only with Host/Origin checks and a 4 MiB body limit; commandsserve,version,list-tools; JSON ready line,/healthz,server.json+server.lockfor dexcli discovery; hot reload ofconnections.json._metaeffect and approval (approval key only when recommended), outcomesunknown/rejected; conformance against the vendored Dex fixture. Identity_metakeys underio.superdurable.connectors/(connector, connection, operation, operationName, connectorVersion). Tool names per the agreed rule (connection name only when a connector has several).io.superdurable.dex/idempotencyKey. Call statedex-connectors-mcp/call-state/v1: one 0600 record per key with flock + inode recheck, fsync ordering, full request identity, frozen retry window, monotonicpossiblyApplied, connection target fingerprint, 30-day retention. A non-idempotent write is sent at most once per key across retries, crashes, concurrency and hot reload; a possibly-applied write is never reportedrejected.--connection connectorId/connectionName[=tool,...]or--connections-config; empty tool lists are usage errors; expected-identity guard ontools/call.authorizationId). Real Dex end-to-end example tracked in connectorsmcp: real Dex end-to-end example for the Connectors MCP server #653.CI and release
connectors-mcp(Go 1.25,make test-connectors-mcpagainst this commit's sdkgo);connectorsmcp/changes no longer trigger the full connector matrix.connectorctl connectors-mcp-inventoryselects connectors whose latest complete Go release exposes agent tools (module path verified);script/release/connectors_mcp_release.pyplans, builds with a hold rule (never silently drops or downgrades a linked connector;withdrawnConnectorsto remove), packages 4 archives (darwin/linux × arm64/amd64) and builds the index.release-connectors-mcpjob: off-main release commit, tagconnectorsmcp/vX.Y.Z, GitHub release withconnectors-mcp-release.complete; failure never blocks connector releases or the catalog. Pages publishesconnectors-mcp/index.json(dex-connectors-mcp/release-index/v1,launchContract1, verified per-asset sha256); a health job after the catalog deploy turns red on fallback, omission, dropped or unverified versions.connectorsmcp/README.md,docs/architecture.md,docs/versioning-and-releases.md; AGENTS.md / .cursor rule mirror for the new tag.No release happens yet: the first
connectorsmcp/v0.1.0is produced once a connector release adopts sdkgo v0.35.0 and generated agent tools.Review
Server: Codex xhigh review plus four re-review rounds (key identity, possibly-applied tracking, retry window, connection rebinding, post-lock snapshots, read path). Release pipeline: Codex high review plus two re-review rounds.
Testing
GOWORK=off go vet ./...andgo test -race -count=2 ./...inconnectorsmcp/: crash injection (kill -9 at protocol points; at most one send per key), concurrency, rebind, hot reload, security, protocol versions, CLI.make test-common,make test-connectors-mcp, connectorctl tests, release script tests (Python),script/ci_workflow_test.py, actionlint.