Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 8 additions & 6 deletions docs/ai-python/content/docs/basics/messages-and-events.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -171,14 +171,16 @@ print(stream.message.usage)

## Validate message history

Before provider calls, the SDK prepares message history by stripping internal
messages, removing non-model parts, repairing invalid tool args to `{}`, and
inserting error results for missing tool calls when possible. Use strict
validation in tests or import pipelines:
`ai.stream` passes message history to the provider as-is. Providers repair
it before the wire call: they strip internal messages, remove non-model
parts, replace invalid tool args with `{}`, and insert error results for
missing tool calls. To fail fast instead — in tests or import pipelines —
validate the history yourself:

```python
from ai.types import integrity
from ai.providers import history_utils


integrity.prepare_messages(messages, mode="strict")
history_utils.validate(messages) # raises IntegrityError on any issue
issues = history_utils.inspect(messages) # or just report them
```
44 changes: 44 additions & 0 deletions docs/ai-python/content/docs/reference/ai.providers/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,47 @@ provider-specific namespaces.
- `OpenAICompatibleProvider`
- `AnthropicCompatibleProvider`
- `GatewayProvider`
- `history_utils`

## ai.providers.history_utils

Message-history repair utilities. `ai.stream` passes history to the
provider as-is; provider implementations call these before converting
messages to their wire format.

```python
from ai.providers import history_utils


repaired = history_utils.repair(messages)
```

`repair` strips internal messages, removes non-model parts, replaces
invalid tool args with `{}`, and inserts error results for missing tool
calls, logging a warning for every fix. It raises `IntegrityError` on
duplicate tool ids and orphaned tool results — those have no safe
automatic fix. The individual fix functions (`drop_internal`,
`fix_tool_args`, `close_orphaned_tool_calls`) each return a new message
list plus the issues they fixed — use them when a provider needs
different choices or the issues themselves — and `check_tool_ids`
detects the unfixable issues without raising.

Validate a history yourself when you want issues to raise instead of
being repaired.

```python
history_utils.validate(messages) # raises IntegrityError on any issue
issues = history_utils.inspect(messages) # or just report them
```

APIs:

- `repair`
- `drop_internal`
- `fix_tool_args`
- `close_orphaned_tool_calls`
- `check_tool_ids`
- `inspect`
- `validate`
- `Issue`
- `IntegrityError`
24 changes: 0 additions & 24 deletions docs/ai-python/content/docs/reference/types.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -32,30 +32,6 @@ Media type helpers:
- `detect_image_media_type`
- `detect_audio_media_type`

## ai.types.integrity

Before provider calls, the SDK prepares message history. It strips internal
messages, removes non-model parts, repairs invalid tool args to `{}`, and
inserts error results for missing tool calls when possible.

```python
from ai.types import integrity


prepared = integrity.prepare_messages(messages)
```

Use strict validation when you want repairable issues to raise.

```python
integrity.prepare_messages(messages, mode="strict")
```

APIs:

- `prepare_messages`
- `IntegrityError`

## ai.types.usage

`Usage` is normalized token usage from a single model call.
Expand Down
9 changes: 3 additions & 6 deletions src/ai/models/core/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@
from typing_extensions import TypeVar

from ... import errors, telemetry, types
from ...types import integrity

if TYPE_CHECKING:
from collections.abc import AsyncGenerator, AsyncIterator, Sequence
Expand Down Expand Up @@ -578,10 +577,9 @@ async def _stream(
)
replay = True
else:
prepared = integrity.prepare_messages(messages)
request = _StreamRequest(
model=model,
messages=prepared,
messages=list(messages),
tools=tools,
output_type=output_type,
params=params,
Expand All @@ -591,7 +589,7 @@ async def _stream(
output_type=cast("type[Any] | None", output_type),
)
data = telemetry.AiStreamSpanData(
model=model.id, messages=prepared, params=params
model=model.id, messages=list(messages), params=params
)
replay = False
# Not set as current: the caller's work while the stream is open
Expand All @@ -614,8 +612,7 @@ async def generate(
params: params_.GenerateParams,
) -> types.messages.Message:
"""Generate a non-streaming response (images, video, etc.)."""
messages = integrity.prepare_messages(messages)
request = _GenerateRequest(model, messages, params)
request = _GenerateRequest(model, list(messages), params)
async with telemetry.span(
telemetry.AiGenerateSpanData(
model=model.id, messages=messages, params=params
Expand Down
2 changes: 2 additions & 0 deletions src/ai/providers/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
"""Provider implementations and factories."""

from . import history_utils
from .ai_gateway import GatewayProvider
from .anthropic import AnthropicCompatibleProvider
from .base import Provider, ProviderProtocol, get_provider
Expand All @@ -12,4 +13,5 @@
"Provider",
"ProviderProtocol",
"get_provider",
"history_utils",
]
4 changes: 2 additions & 2 deletions src/ai/providers/ai_gateway/protocol.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from ... import types
from ...models import core
from ...models.core import params as params_
from .. import base
from .. import base, history_utils
from . import client as gateway_client
from . import errors
from . import params as gateway_params
Expand Down Expand Up @@ -168,7 +168,7 @@ async def _messages_to_prompt(
"""Convert ``Message`` list to the v3 prompt wire format."""
result: list[dict[str, Any]] = []

for msg in messages:
for msg in history_utils.repair(messages):
match msg.role:
case "system":
text = "".join(
Expand Down
4 changes: 2 additions & 2 deletions src/ai/providers/anthropic/protocol.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
from ... import types
from ...models.core import params as params_
from ...types import events
from .. import base
from .. import base, history_utils
from . import _sdk, errors
from . import tools as anthropic_tools

Expand Down Expand Up @@ -229,7 +229,7 @@ async def _messages_to_anthropic(
system_prompt: str | None = None
result: list[dict[str, Any]] = []

for msg in messages:
for msg in history_utils.repair(messages):
match msg.role:
case "system":
system_prompt = "".join(
Expand Down
Loading
Loading