Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/api/rlm.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ RLM(
#### `backend`
{: .no_toc }

**Type:** `Literal["openai", "portkey", "openrouter", "vllm", "litellm", "anthropic"]`
**Type:** `Literal["openai", "anthropic", "azure_anthropic", "azure_openai", "portkey", "openrouter", "vllm", "litellm", "gemini", "vercel"]`
**Default:** `"openai"`

The LM provider backend to use for the root model.
Expand Down Expand Up @@ -88,6 +88,8 @@ Configuration passed to the LM client. Required fields vary by backend:
|:--------|:---------|:---------|
| `openai` | `model_name` | `api_key`, `base_url` |
| `anthropic` | `model_name` | `api_key` |
| `azure_anthropic` | `model_name` | `api_key`, `resource`, `base_url` |
| `azure_openai` | `model_name` | `api_key`, `azure_endpoint`, `azure_deployment`, `api_version` |
| `portkey` | `model_name`, `api_key` | `base_url` |
| `openrouter` | `model_name` | `api_key` |
| `vllm` | `model_name`, `base_url` | — |
Expand Down
27 changes: 27 additions & 0 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -247,6 +247,33 @@ rlm = RLM(
)
```

### Azure Anthropic (Foundry)

For Anthropic models hosted on [Azure AI Foundry](https://ai.azure.com/).
Uses the same `ANTHROPIC_FOUNDRY_*` environment variable convention as
[Claude Code](https://docs.anthropic.com/en/docs/claude-code):

```bash
# .env
ANTHROPIC_FOUNDRY_API_KEY=...
ANTHROPIC_FOUNDRY_RESOURCE=ml-platform-openai-stg-useast-2
# Or use a project-scoped endpoint instead of RESOURCE:
# ANTHROPIC_FOUNDRY_BASE_URL=https://res.services.ai.azure.com/api/projects/my-proj
```

```python
rlm = RLM(
backend="azure_anthropic",
backend_kwargs={
"model_name": "claude-opus-4-6",
# Credentials read from ANTHROPIC_FOUNDRY_* env vars.
# Or pass explicitly:
# "resource": "ml-platform-openai-stg-useast-2",
# "api_key": "...",
},
)
```

### Portkey (Router)

```python
Expand Down
28 changes: 27 additions & 1 deletion docs/src/app/backends/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ export default function BackendsPage() {

<p className="text-muted-foreground mb-6">
<p>
RLMs natively support a wide range of language model providers, including <code>OpenAI</code>, <code>Anthropic</code>, <code>Portkey</code>, <code>OpenRouter</code>, and <code>LiteLLM</code>. Additional providers can be supported with minimal effort. The <code>backend_kwargs</code> are named arguments passed directly to the backend client.
RLMs natively support a wide range of language model providers, including <code>OpenAI</code>, <code>Anthropic</code>, <code>Azure Anthropic (Foundry)</code>, <code>Azure OpenAI</code>, <code>Portkey</code>, <code>OpenRouter</code>, and <code>LiteLLM</code>. Additional providers can be supported with minimal effort. The <code>backend_kwargs</code> are named arguments passed directly to the backend client.
</p>
</p>

Expand Down Expand Up @@ -36,6 +36,32 @@ export default function BackendsPage() {

<hr className="my-8 border-border" />

<h2 className="text-2xl font-semibold mb-4">Azure Anthropic (Foundry)</h2>
<p className="text-muted-foreground mb-4">
For Anthropic models hosted on{" "}
<a href="https://ai.azure.com/" className="text-primary underline font-medium" target="_blank" rel="noopener noreferrer">Azure AI Foundry</a>.
Uses the same <code>ANTHROPIC_FOUNDRY_*</code> environment variable convention as{" "}
<a href="https://docs.anthropic.com/en/docs/claude-code" className="text-primary underline font-medium" target="_blank" rel="noopener noreferrer">Claude Code</a>,
so if you already have those env vars set, this backend picks them up automatically.
</p>
<CodeBlock language="bash" code={`# Set Foundry credentials (same vars Claude Code uses)
export ANTHROPIC_FOUNDRY_API_KEY="your-key"
export ANTHROPIC_FOUNDRY_RESOURCE="ml-platform-openai-stg-useast-2"
# Or use a project-scoped endpoint instead of RESOURCE:
# export ANTHROPIC_FOUNDRY_BASE_URL="https://res.services.ai.azure.com/api/projects/my-proj"`} />
<CodeBlock code={`rlm = RLM(
backend="azure_anthropic",
backend_kwargs={
"model_name": "claude-opus-4-6",
# Credentials read from ANTHROPIC_FOUNDRY_* env vars.
# Or pass explicitly:
# "resource": "ml-platform-openai-stg-useast-2",
# "api_key": "...",
},
)`} />

<hr className="my-8 border-border" />

<h2 className="text-2xl font-semibold mb-4">Portkey</h2>
<p className="text-muted-foreground mb-4">
<a href="https://portkey.ai/docs/api-reference/sdk/python" className="text-primary underline font-medium" target="_blank" rel="noopener noreferrer">Portkey</a> is a client for routing to hundreds of different open and closed frontier models.
Expand Down
6 changes: 5 additions & 1 deletion rlm/clients/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,11 @@ def get_client(
from rlm.clients.azure_openai import AzureOpenAIClient

return AzureOpenAIClient(**backend_kwargs)
elif backend == "azure_anthropic":
from rlm.clients.azure_anthropic import AzureAnthropicClient

return AzureAnthropicClient(**backend_kwargs)
else:
raise ValueError(
f"Unknown backend: {backend}. Supported backends: ['openai', 'vllm', 'portkey', 'openrouter', 'litellm', 'anthropic', 'azure_openai', 'gemini', 'vercel']"
f"Unknown backend: {backend}. Supported backends: ['openai', 'vllm', 'portkey', 'openrouter', 'litellm', 'anthropic', 'azure_openai', 'azure_anthropic', 'gemini', 'vercel']"
)
164 changes: 164 additions & 0 deletions rlm/clients/azure_anthropic.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
import os
from collections import defaultdict
from typing import Any

import anthropic
from dotenv import load_dotenv

from rlm.clients.base_lm import BaseLM
from rlm.core.types import ModelUsageSummary, UsageSummary

load_dotenv()

DEFAULT_FOUNDRY_API_KEY = os.getenv("ANTHROPIC_FOUNDRY_API_KEY")
DEFAULT_FOUNDRY_RESOURCE = os.getenv("ANTHROPIC_FOUNDRY_RESOURCE")
DEFAULT_FOUNDRY_BASE_URL = os.getenv("ANTHROPIC_FOUNDRY_BASE_URL")

ANTHROPIC_PATH_SUFFIX = "/anthropic/v1"


def _resolve_base_url(base_url: str | None, resource: str | None) -> str:
"""Derive the Foundry base URL from explicit URL or resource name.

Accepts either:
- A resource name → https://{resource}.services.ai.azure.com/anthropic/v1
- A Foundry endpoint URL (resource- or project-scoped)
e.g. https://res.services.ai.azure.com
https://res.services.ai.azure.com/api/projects/my-proj

The /anthropic/v1 suffix is always appended (unless already present).
"""
if base_url:
url = base_url.rstrip("/")
if not url.endswith(ANTHROPIC_PATH_SUFFIX):
url = url + ANTHROPIC_PATH_SUFFIX
return url
if resource:
return f"https://{resource}.services.ai.azure.com{ANTHROPIC_PATH_SUFFIX}"
raise ValueError(
"Azure Anthropic Foundry endpoint is required. "
"Set ANTHROPIC_FOUNDRY_BASE_URL, ANTHROPIC_FOUNDRY_RESOURCE, "
"or pass base_url/resource as an argument."
)


class AzureAnthropicClient(BaseLM):
"""
LM Client for running Anthropic models hosted on Azure AI Foundry.

Follows the ANTHROPIC_FOUNDRY_* env-var convention used by Claude Code:
ANTHROPIC_FOUNDRY_API_KEY - API key
ANTHROPIC_FOUNDRY_RESOURCE - Azure resource name (derives base URL)
ANTHROPIC_FOUNDRY_BASE_URL - Explicit base URL (overrides resource)
"""

def __init__(
self,
api_key: str | None = None,
model_name: str | None = None,
base_url: str | None = None,
resource: str | None = None,
max_tokens: int = 4096,
**kwargs,
):
super().__init__(model_name=model_name, **kwargs)

if api_key is None:
api_key = DEFAULT_FOUNDRY_API_KEY

if base_url is None:
base_url = DEFAULT_FOUNDRY_BASE_URL

if resource is None:
resource = DEFAULT_FOUNDRY_RESOURCE

resolved_url = _resolve_base_url(base_url, resource)

self.client = anthropic.Anthropic(api_key=api_key, base_url=resolved_url)
self.async_client = anthropic.AsyncAnthropic(api_key=api_key, base_url=resolved_url)
self.model_name = model_name
self.max_tokens = max_tokens

# Per-model usage tracking
self.model_call_counts: dict[str, int] = defaultdict(int)
self.model_input_tokens: dict[str, int] = defaultdict(int)
self.model_output_tokens: dict[str, int] = defaultdict(int)
self.model_total_tokens: dict[str, int] = defaultdict(int)

def completion(self, prompt: str | list[dict[str, Any]], model: str | None = None) -> str:
messages, system = self._prepare_messages(prompt)

model = model or self.model_name
if not model:
raise ValueError("Model name is required for Azure Anthropic client.")

kwargs = {"model": model, "max_tokens": self.max_tokens, "messages": messages}
if system:
kwargs["system"] = system

response = self.client.messages.create(**kwargs)
self._track_cost(response, model)
return response.content[0].text

async def acompletion(
self, prompt: str | list[dict[str, Any]], model: str | None = None
) -> str:
messages, system = self._prepare_messages(prompt)

model = model or self.model_name
if not model:
raise ValueError("Model name is required for Azure Anthropic client.")

kwargs = {"model": model, "max_tokens": self.max_tokens, "messages": messages}
if system:
kwargs["system"] = system

response = await self.async_client.messages.create(**kwargs)
self._track_cost(response, model)
return response.content[0].text

def _prepare_messages(
self, prompt: str | list[dict[str, Any]]
) -> tuple[list[dict[str, Any]], str | None]:
"""Prepare messages and extract system prompt for Anthropic API."""
system = None

if isinstance(prompt, str):
messages = [{"role": "user", "content": prompt}]
elif isinstance(prompt, list) and all(isinstance(item, dict) for item in prompt):
messages = []
for msg in prompt:
if msg.get("role") == "system":
system = msg.get("content")
else:
messages.append(msg)
else:
raise ValueError(f"Invalid prompt type: {type(prompt)}")

return messages, system

def _track_cost(self, response: anthropic.types.Message, model: str):
self.model_call_counts[model] += 1
self.model_input_tokens[model] += response.usage.input_tokens
self.model_output_tokens[model] += response.usage.output_tokens
self.model_total_tokens[model] += response.usage.input_tokens + response.usage.output_tokens

self.last_prompt_tokens = response.usage.input_tokens
self.last_completion_tokens = response.usage.output_tokens

def get_usage_summary(self) -> UsageSummary:
model_summaries = {}
for model in self.model_call_counts:
model_summaries[model] = ModelUsageSummary(
total_calls=self.model_call_counts[model],
total_input_tokens=self.model_input_tokens[model],
total_output_tokens=self.model_output_tokens[model],
)
return UsageSummary(model_usage_summaries=model_summaries)

def get_last_usage(self) -> ModelUsageSummary:
return ModelUsageSummary(
total_calls=1,
total_input_tokens=self.last_prompt_tokens,
total_output_tokens=self.last_completion_tokens,
)
1 change: 1 addition & 0 deletions rlm/core/types.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
"litellm",
"anthropic",
"azure_openai",
"azure_anthropic",
"gemini",
]
EnvironmentType = Literal["local", "docker", "modal", "prime", "daytona", "e2b"]
Expand Down
Loading