Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 66 additions & 23 deletions modules/ai-gateway/pages/connect-agent.adoc
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
= Connect Your Agent
:description: Point your application or AI agent at an AI Gateway provider's proxy URL. Covers the URL shape, the local auth flow with the `rpk ai` plugin, the OIDC client-credentials flow for CI, and SDK examples for OpenAI, Anthropic, Google AI, AWS Bedrock, and OpenAI-compatible endpoints.
:description: Point your application or AI agent at an AI Gateway provider's proxy URL. Covers the URL shape, the local development workflow with `rpk ai`, the OIDC client-credentials flow for CI and application code, and SDK examples for OpenAI, Anthropic, Google AI, AWS Bedrock, and OpenAI-compatible endpoints.
:page-topic-type: how-to
:personas: app_developer
:page-aliases: redpanda-cloud:ai-agents:ai-gateway/builders/connect-your-agent.adoc
:learning-objective-1: Construct the proxy URL for an LLM provider you have configured
:learning-objective-2: Authenticate to AI Gateway using the `rpk ai` plugin for local development or OIDC client credentials for CI and programmatic clients
:learning-objective-2: Authenticate to AI Gateway with `rpk` for local development or with OIDC client credentials for CI and programmatic clients
:learning-objective-3: Send requests through the proxy URL with the SDK of your choice

This guide shows how to connect your glossterm:AI agent[] or application to AI Gateway. You'll construct the proxy URL for a provider you have already created, authenticate (with the `rpk ai` plugin for local development or with OIDC client credentials for CI), and send your first request with the SDK of your choice.
This guide shows how to connect your glossterm:AI agent[] or application to AI Gateway. You'll construct the proxy URL for a provider you have already created, authenticate (with `rpk cloud login` for local development or with OIDC client credentials for CI and application code), and send your first request with the SDK of your choice.

After completing this guide, you will be able to:

Expand All @@ -18,7 +18,7 @@ After completing this guide, you will be able to:
== Prerequisites

* A configured LLM provider. If you haven't created one yet, see xref:configure-provider.adoc[Configure an LLM provider].
* For local development: nothing else; you'll install the `rpk ai` plugin in the next section.
* For local development: nothing else; you'll install `rpk ai` in the next section.
Comment thread
micheleRP marked this conversation as resolved.
Outdated
* For CI or programmatic clients: a Redpanda Cloud service account with OIDC client credentials. See xref:redpanda-cloud:security:cloud-authentication.adoc[Authenticate to Redpanda Cloud].
+
// TODO: confirm whether ADP hosts its own service-account IAM post-standalone, or continues to share Redpanda Cloud Organization IAM.
Expand All @@ -41,41 +41,84 @@ AI Gateway forwards the request to the upstream provider, attaches the configure

TIP: The provider detail page generates ready-to-run snippets pre-filled with the correct proxy URL and paths. When in doubt, copy from the *Connect your app* section there.

// Updated for PRs #30273 / #30327 / #30360 (rpk ai managed plugin).
[[authenticate-with-rpk-ai]]
[[authenticate-with-rpai]]
== Authenticate with `rpk ai` (recommended for local development)
== Use `rpk ai` for local development

The `rpk ai` plugin is distributed through `rpk`'s plugin manager. The provider detail page surfaces an *Install* card with copy-pasteable steps. The flow is the same for every provider type:
`rpk ai` is the Redpanda AI CLI. Use it to manage AI Gateway resources (LLM providers, MCP servers, OAuth providers) and call MCP tools from the command line. Authentication for `rpk ai` is owned by `rpk cloud login`; the active AI Gateway URL comes from your active rpk cloud profile.

. Install the plugin:
. Install `rpk ai`:
+
[source,bash]
----
rpk plugin install ai
rpk ai install
----
+
Update later with `rpk ai upgrade`; remove with `rpk ai uninstall`.

. Log in with the gateway URL from the provider's *Connection* card:
. Log in to Redpanda Cloud:
+
[source,bash]
----
rpk ai auth login --server https://aigw.<cluster-id>.clusters.rdpa.co
rpk cloud login
----
+
This caches a cloud token in `~/.config/rpk/rpk.yaml`. `rpk ai` reads this token automatically on every invocation.

. Point your SDK at the proxy URL and let `rpk ai auth token` mint a fresh token on each call. Set environment variables:
. Select an rpk cloud profile that points at a cluster with AI Gateway v2 attached. The AI Gateway URL is cached on the profile when you create it.
Comment thread
micheleRP marked this conversation as resolved.
Outdated
+
[source,bash]
----
export PROXY_URL="<your-gateway-base>/llm/v1/providers/<provider-name>"
export OPENAI_API_KEY="$(rpk ai auth token)" # or ANTHROPIC_API_KEY, etc.
rpk profile use <profile-name>
# or, to switch the cluster the active profile points at:
rpk cloud cluster use <cluster-id>
----

`rpk ai auth token` returns a short-lived OIDC access token. Refresh by running it again: most users wire it into a wrapper script or shell function.
. Verify the connection:
+
[source,bash]
----
rpk ai llm list
----

TIP: The plugin supports named profiles for pointing at multiple gateways. Run `rpk ai profile create <name> --dataplane-url <gateway-url> --auth-mode device` to create one, then `rpk ai profile use <name>` to switch. See `rpk ai profile --help` for the full set of subcommands.
If the cached cloud token has expired, `rpk ai` returns a 401 with a hint to rerun `rpk cloud login`.

[TIP]
====
To target a specific gateway URL for a single invocation (for example, when running against a staging gateway without switching profiles), pass `--rpai-endpoint`:

[source,bash]
----
rpk ai --rpai-endpoint https://aigw.<cluster-id>.clusters.rdpa.co llm list
----

You can also export `RPAI_ENDPOINT` to override for the shell session.
====

// TODO(rpk-ai): rpai suppresses auth/profile subtrees in plugin mode today (cloudv2 apps/rpai/internal/cmd/root.go:127-135). If that changes, document `rpk ai auth` and `rpk ai profile` here.

=== Environment variables

`rpk ai` honors these environment variables:

[cols="1,3"]
|===
|Variable |Purpose

|`RPAI_TOKEN`
|Bearer token for the gateway. Normally injected automatically from your cached `rpk cloud login` token; set explicitly to override.

|`RPAI_ENDPOINT`
|AI Gateway URL. Normally resolved from your active rpk cloud profile; set explicitly to override.

|`RPAI_PROFILE`, `RPAI_CONFIG`, `RPAI_VERBOSE`, `RPAI_FORMAT`
|Map to `--rpai-profile`, `--rpai-config`, `--rpai-verbose`, `--format`. Long flag names are renamed under `rpk ai` to avoid collision with `rpk`'s globals; short flags (`-p`, `-c`, `-v`, `-o`) are unchanged.
|===

== Authenticate with OIDC client credentials (CI and programmatic)

When the `rpk ai` plugin isn't available (CI runners, server-side processes, headless agents), use the OIDC `client_credentials` grant directly. Values are surfaced on the provider's *Connection* card; defaults at the time of writing are below.
For application code, CI runners, server-side processes, and headless agents, use the OIDC `client_credentials` grant directly. This is the canonical authentication path for SDK-style usage; `rpk ai` is for command-line workflows, not for embedding in application code. Values are surfaced on the provider's *Connection* card; defaults at the time of writing are below.

[cols="1,2", options="header"]
|===
Expand Down Expand Up @@ -175,7 +218,7 @@ IMPORTANT: Your client is responsible for refreshing tokens before they expire.
* Proactively refresh at ~80% of the token's TTL to avoid failed requests.
* `authlib` (Python) handles renewal automatically when you pass `token_endpoint` to `OAuth2Session`.
* For other languages, cache the token and its expiry, then request a new token before the current one expires.
* If you're using `rpk ai`, just rerun `rpk ai auth token`: it handles refresh against the same OIDC endpoint.
* For SDK code, refresh OIDC client-credentials tokens through your client library (see the `authlib` example above).

== Send requests with your SDK

Expand All @@ -184,7 +227,7 @@ The examples in this section assume you've set:
[source,bash]
----
export PROXY_URL="<your-gateway-base>/llm/v1/providers/<provider-name>"
export AUTH_TOKEN="$(rpk ai auth token)" # or an OIDC access token from above
export AUTH_TOKEN="<oidc-access-token>" # from the client_credentials flow above
----

[tabs]
Expand All @@ -198,7 +241,7 @@ from openai import OpenAI

client = OpenAI(
base_url=os.environ["PROXY_URL"], # .../llm/v1/providers/my-openai
api_key=os.environ["AUTH_TOKEN"], # rpk ai or OIDC access token
api_key=os.environ["AUTH_TOKEN"], # OIDC access token
)

response = client.chat.completions.create(
Expand All @@ -219,7 +262,7 @@ from anthropic import Anthropic

client = Anthropic(
base_url=os.environ["PROXY_URL"], # .../llm/v1/providers/my-anthropic
auth_token=os.environ["AUTH_TOKEN"], # rpk ai or OIDC access token
auth_token=os.environ["AUTH_TOKEN"], # OIDC access token
)

message = client.messages.create(
Expand Down Expand Up @@ -258,7 +301,7 @@ Gemini authenticates with the `x-goog-api-key` header, not `Authorization: Beare

AWS Bedrock::
+
Bedrock is different: SigV4 signing is performed *server-side* by AI Gateway using the credentials on the provider. Your client only needs to call the proxy URL with an `rpk ai` or OIDC token.
Bedrock is different: SigV4 signing is performed *server-side* by AI Gateway using the credentials on the provider. Your client only needs to call the proxy URL with an OIDC access token.
+
[source,python]
----
Expand Down Expand Up @@ -355,7 +398,7 @@ AI Gateway returns standard HTTP status codes. The upstream provider's error bod
== Best practices

* *Use environment variables* for the proxy URL and token; never hard-code them.
* *Wrap `rpk ai auth token`* in a script or shell function so refresh is invisible to your SDK code.
* *Refresh OIDC tokens through your client library* so refresh is invisible to your SDK code (`authlib` for Python, `openid-client` for Node.js, etc.).
* *Implement retry with exponential backoff* for 5xx and timeout conditions.
* *Respect `Retry-After`* on 429 responses.
* *Rotate service account credentials* on a schedule your organization accepts.
Expand All @@ -365,7 +408,7 @@ AI Gateway returns standard HTTP status codes. The upstream provider's error bod

=== 401 Unauthorized

* If you're using `rpk ai`: rerun `rpk ai auth login` to refresh the session, then `rpk ai auth token` to mint a new token.
* If you're using `rpk ai`: rerun `rpk cloud login` to refresh the cached cloud token. Token expiry surfaces as a 401 with this hint in the error.
* If you're using OIDC client credentials: check the token hasn't expired and refresh it. Verify the audience is `cloudv2-production.redpanda.cloud` and the `Authorization` header is formatted `Bearer <token>`.
* For Gemini: ensure the token is sent as `x-goog-api-key`, not `Authorization`.
* For Anthropic with passthrough: ensure the client is sending a valid Anthropic `Authorization` header.
Expand Down
2 changes: 1 addition & 1 deletion modules/ai-gateway/pages/overview.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ Use the provider's own SDK: OpenAI, Anthropic, Google AI, AWS Bedrock, or any Op

=== Managed authentication

Applications authenticate to ADP with OIDC service accounts instead of long-lived provider API keys. Service accounts use the same role and audit model as every other ADP resource, and mint short-lived tokens that are easy to revoke. The recommended local flow uses the `rpk ai` plugin for token refresh; CI and programmatic clients use the OIDC client-credentials grant directly. See xref:connect-agent.adoc[Connect your agent].
Applications authenticate to ADP with OIDC service accounts instead of long-lived provider API keys. Service accounts use the same role and audit model as every other ADP resource, and mint short-lived tokens that are easy to revoke. For local command-line workflows, use `rpk cloud login` to authenticate and `rpk ai` to talk to the gateway. CI and programmatic clients use the OIDC client-credentials grant directly. See xref:connect-agent.adoc[Connect your agent].

=== Per-provider observability

Expand Down
9 changes: 4 additions & 5 deletions modules/mcp/pages/create-server.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ A populated tools list confirms that the connection works and credentials resolv

== Create from the CLI

The `rpk ai` plugin offers a non-UI path for the same create flow. Useful for scripting and CI.
`rpk ai` offers a non-UI path for the same create flow. Useful for scripting and CI.

[source,bash]
----
Expand Down Expand Up @@ -185,12 +185,11 @@ rpk ai mcp update my-ramp \

|`--user-oauth-scopes`
|Comma-separated scopes the server requires. Provide every scope any tool may need; user re-consent is required if scopes change later.

|`--auth-config`
|Alternative when a server needs an auth shape that doesn't map to the convenience flags. Takes a JSON blob matching the appropriate auth oneof variant.
|===

The CLI uses your active `rpk ai` profile for the gateway URL and authentication.
// TODO(rpk-ai): --auth-config does not exist on `rpk ai mcp create` today (verified against cloudv2/apps/rpai/internal/cmd/mcp/cmd.go on 2026-05-05). For non-convenience auth shapes, encode them inside `--managed-config` JSON. Re-add this row if a flag lands later.

`rpk ai` resolves the gateway URL from your active rpk cloud profile and reads the cached `rpk cloud login` token.

== Edit, disable, and delete a server

Expand Down
48 changes: 33 additions & 15 deletions modules/mcp/pages/oauth-providers.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -171,16 +171,19 @@ The provider appears in the *OAuth providers* list.

== Register from the CLI

Use the `rpk ai` plugin to script provider registration:
Use `rpk ai` to script provider registration:

// TODO(rpk-ai): aliases (oauth-provider, mcp-server, llm-provider) are kept for one release per cloudv2/apps/rpai/CLAUDE.md. Verified canonical command is `rpk ai oauth create` on 2026-05-05; flip examples cleanly to canonical names if the alias gets dropped.

[source,bash]
----
rpk ai oauth-provider create --name ramp \
--type oauth2 \
rpk ai oauth create \
--name ramp \
--display-name "Ramp" \
--auth-endpoint "https://app.ramp.com/v1/authorize" \
--token-endpoint "https://api.ramp.com/developer/v1/token" \
--client-id "$RAMP_CLIENT_ID" \
--client-secret-ref RAMP_CLIENT_SECRET \
--auth-url "https://app.ramp.com/v1/authorize" \
--token-url "https://api.ramp.com/developer/v1/token" \
--scopes "transactions:read,cards:read,users:read"
----

Expand All @@ -189,25 +192,40 @@ rpk ai oauth-provider create --name ramp \
|Flag |Notes

|`--name`
|Resource name. Lowercase letters, numbers, hyphens. Immutable.
|Resource name. Lowercase letters, numbers, hyphens. Immutable. Required.

|`--display-name`
|Human-readable display name shown in the UI. Required.

|`--auth-endpoint`
|OAuth authorization endpoint URL. Required.

|`--type`
|`oauth2` for the standard authorization-code flow.
|`--token-endpoint`
|OAuth token endpoint URL. Required.

|`--client-id`
|Client ID from the upstream OAuth app.
|Client ID from the upstream OAuth app. Required.

|`--client-secret-ref`
|Secret-store reference (`UPPER_SNAKE_CASE`).

|`--auth-url`
|Authorization endpoint.

|`--token-url`
|Token endpoint.

|`--scopes`
|Comma-separated scope list.

|`--grant-types`
|Grant types: `browser-consent` (default), `token-exchange`. Comma-separated.

|`--token-auth-method`
|Token-endpoint authentication method: `client-secret-basic` (default), `client-secret-post`, `none`.

|`--pkce`
|Require PKCE for authorization code grants.

|`--revocation-endpoint`
|OAuth token revocation endpoint URL.

|`--enabled`
|Whether the provider is enabled (default `true`).
|===

== Attach to an MCP server
Expand Down
4 changes: 2 additions & 2 deletions modules/mcp/pages/test-tools.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ The Session panel keeps a running history of every call you've made through the

== Test from the CLI

The `rpk ai` plugin offers a non-UI path for the same tool calls. Use it when scripting smoke tests or running checks from CI.
`rpk ai` offers a non-UI path for the same tool calls. Use it when scripting smoke tests or running checks from CI.

[source,bash]
----
Expand All @@ -118,7 +118,7 @@ rpk ai mcp tools call <server-name> <tool-name> --input '{"arg1":"value"}'
rpk ai mcp get <server-name>
----

The CLI uses your active `rpk ai` profile for the gateway URL and authentication. See xref:ai-gateway:connect-agent.adoc[Connect your agent] for installation and profile setup.
`rpk ai` resolves the gateway URL from your active rpk cloud profile and reads the cached `rpk cloud login` token. See xref:ai-gateway:connect-agent.adoc[Connect your agent] for installation and profile setup.

== Out of scope

Expand Down