Skip to content
123 changes: 123 additions & 0 deletions docs/reasoning-levels.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
# Reasoning levels

UnifiedLLM exposes the choices a configured route supports. Model names do not
determine those choices: registry YAML maps each label to exact request parameters.
Selecting a label applies `params.update(level_settings)` before dispatch.

```python
from pathlib import Path

from nooa.unifiedllm import get_llm_client
from nooa.unifiedllm.registry import reload_registry

# From the repository root; this example registry is not loaded automatically.
reload_registry(Path("examples/reasoning_levels/llm_config.yaml"))
client = get_llm_client("gpt-5.6-sol")
print(client.reasoning_levels) # tuple of labels; None = unknown, () = unsupported
print(client.reasoning_default) # documented default, or None when unknown
response = await client.acall(messages, reasoning_level="high")
```

The constructor also accepts `reasoning_level` as a persistent selection. A
per-call selection overrides it; explicit `reasoning_level=None` uses the raw
base configuration for that call. `reasoning_default` is metadata only: declaring
it does not add parameters, change costs or override existing provider settings.

## Registry declarations

```yaml
models:
my-route:
model_name: openai/my-model
reasoning: {effort: medium, context: all_turns}
reasoning_default: medium
reasoning_levels:
low: {reasoning: {effort: low, context: all_turns}}
medium: {reasoning: {effort: medium, context: all_turns}}
high: {reasoning: {effort: high, context: all_turns}}
```

Write the complete nested block for each level. There is no deep merge or
inheritance: selecting a level replaces the base value at each key it sets.
The declarations are trusted configuration, just like the rest of the registry;
they are not restricted to a list of provider fields maintained by NOOA.
Client routing and framework controls are reserved: `model`, `api_base`,
`base_url`, `api_key`, `custom_llm_provider`, `client`, `messages`, `input`, `extra_body`,
and the three `reasoning_*` configuration fields. They cannot appear inside a
level's settings. A level changes effort, not the endpoint, credentials or history.

Omit `reasoning_levels` (or use null) when support is unknown. An empty mapping
explicitly declares selection unsupported. Unknown support, unsupported selection
and invalid labels produce distinct errors; an invalid label lists valid choices.
Without a managed selection, raw provider parameters continue working as before.

A selected level cannot be combined with a per-call setting of the same key,
including inside `extra_body`. Choose the label or the raw settings, not both.
Constructor defaults in `extra_body` are replaced by the selected settings,
just like top-level defaults. Unrelated defaults are preserved.
Declarations belong on the constructor, not per-call kwargs or `extra_body`.
Changing model or endpoint while using a managed level requires a new client:
one route's declared choices must not be applied to another route.
Supplying an SDK client per call is also rejected with a managed level because
that client can choose a different endpoint. When overriding a registry alias's
route or client type, inherited levels, default and selection are cleared;
declare replacement levels explicitly, or leave support unknown.

## Where the data comes from

The [example registry](../examples/reasoning_levels/llm_config.yaml) illustrates
three request shapes, based on the [GPT-5.6 Sol model documentation](https://developers.openai.com/api/docs/models/gpt-5.6-sol),
[Claude effort documentation](https://platform.claude.com/docs/en/build-with-claude/effort)
and [Gemini's OpenAI-compatible API](https://ai.google.dev/gemini-api/docs/openai).
Its endpoints are placeholders. Replace them, the model IDs and the declared
choices with settings for your own route before making calls. It is not
auto-loaded. Load your configured file with `reload_registry(Path(...))`.
Private endpoint settings and credentials belong in private configuration, not
this public example.

Provider documentation describes provider APIs, not all gateway routes. Mocked
HTTP tests check that the example's settings survive the installed transport;
the opt-in live test checks your configured route's acceptance, not reasoning quality or every
level's behavior. Do not infer support merely from a successful HTTP response
if a gateway silently ignores parameters.

Do not enable LiteLLM's global `drop_params` when verifying a declaration: it can
discard fields for gateway IDs it does not recognize, even with per-call
`drop_params=False`. The HTTP tests explicitly disable that global flag and check
the serialized fields. NOOA does not change process-global SDK configuration.

LangChain/Pi data can inform maintenance, but neither is a runtime dependency or
an automatic build input. Updating a route means reviewing its small declaration
and request tests, rather than importing hundreds of profiles.

## Scope and architecture

- `unifiedllm/reasoning.py` validates declarations and applies the chosen settings.
The same function serves synchronous and asynchronous Chat and Responses calls.
- Registry fields are passed into UnifiedLLM and consumed before provider dispatch.
Renderers, events and middleware do not translate reasoning levels.
- No selection changes existing behavior. Stored reasoning, compatibility gates,
session archives and replay remain unchanged. This is not a retention toggle.
- Effort labels are provider-local, not comparable units of intelligence or cost.
Changing effort can invalidate a provider's cached prefix. This PR does not add
cache-preserving mid-turn steering, TUI controls, or selection persistence.

## Tests

Run `uv run pytest tests/unifiedllm/test_reasoning_levels.py tests/unifiedllm/test_reasoning_levels_wire.py`.
The tests check configuration ownership, invalid selections, route changes,
unchanged defaults and the serialized HTTP requests for the example routes.

For paid probes, configure registry aliases with a `low` level and credentials
through their normal `api_key_env` settings. Then opt in and name the aliases:

```sh
NOOA_RUN_REASONING_LEVELS_LIVE=1 NOOA_REASONING_TEST_MODELS=my-route uv run pytest \
tests/integration/test_reasoning_levels_live.py -m integration -q -s
```

The alias list is comma-separated. Missing aliases are skipped. Each configured
alias gets one request capped at 256 output tokens, with retries disabled.
The test checks outgoing settings and route acceptance, not that every effort
label changes model behavior. Provider-specific results belong with the
configuration used to run them.
44 changes: 44 additions & 0 deletions examples/reasoning_levels/llm_config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Illustrative request shapes, not a built-in or verified model catalog.
# Replace the placeholder endpoint, model IDs and levels with your route's settings.
models:
gpt-5.6-sol:
model_name: openai/gpt-5.6-sol
client_type: responses
api_base: https://gateway.example.com/v1
api_key_env: MODEL_API_KEY
store: false
include: [reasoning.encrypted_content]
reasoning: {effort: medium}
# https://developers.openai.com/api/docs/models/gpt-5.6-sol
reasoning_default: medium
reasoning_levels:
none: {reasoning: {effort: none}}
low: {reasoning: {effort: low}}
medium: {reasoning: {effort: medium}}
high: {reasoning: {effort: high}}
xhigh: {reasoning: {effort: xhigh}}
max: {reasoning: {effort: max}}

claude-sonnet-5:
model_name: anthropic/claude-sonnet-5
api_base: https://gateway.example.com
api_key_env: MODEL_API_KEY
# https://platform.claude.com/docs/en/build-with-claude/effort
reasoning_default: high
reasoning_levels:
low: {thinking: {type: adaptive}, output_config: {effort: low}}
medium: {thinking: {type: adaptive}, output_config: {effort: medium}}
high: {thinking: {type: adaptive}, output_config: {effort: high}}
max: {thinking: {type: adaptive}, output_config: {effort: max}}

gemini-3.1-pro-preview:
model_name: openai/gemini-3.1-pro-preview
api_base: https://gateway.example.com/v1
api_key_env: MODEL_API_KEY
allowed_openai_params: [reasoning_effort]
# https://ai.google.dev/gemini-api/docs/openai#thinking
reasoning_default: high
reasoning_levels:
low: {reasoning_effort: low}
medium: {reasoning_effort: medium}
high: {reasoning_effort: high}
62 changes: 61 additions & 1 deletion skills/nooa-agent-authoring/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: nooa-agent-authoring
description: Author agents with NVIDIA-labs Object Oriented Agents (NOOA). Use when writing or modifying an Agent subclass, agentic methods (ellipsis bodies), docstring prompts, structured output contracts, strategy selection (CodeAct/Predict), visibility control, orchestrators, or subagent composition.
description: Author agents with NVIDIA-labs Object Oriented Agents (NOOA). Use when configuring model registry entries or reasoning levels, writing or modifying an Agent subclass, agentic methods (ellipsis bodies), docstring prompts, structured output contracts, strategy selection (CodeAct/Predict), visibility control, orchestrators, or subagent composition.
compatibility: 'Python >= 3.12, uv, nooa package (CLI: nooa)'
---

Expand Down Expand Up @@ -80,6 +80,66 @@ Without a rendered boundary, direct callers cache only leading instructions.
[stable-prefix caching](../../docs/stable-prefix-caching.md) for migration and
direct-call examples.

### Configuring reasoning levels

Declare choices for the **configured route**, not a model family inferred from
its name. Each `reasoning_levels` label maps to exact request parameters;
selection replaces whole top-level values, with no nested merge. Repeat any
nested settings that must survive a level change.

For a Responses route that accepts this request shape, adapt this
`llm_config.yaml` example (replace the placeholder model and configure its key):

```yaml
models:
my-route:
model_name: openai/your-model
client_type: responses
reasoning: {effort: medium, summary: auto}
reasoning_default: medium
reasoning_levels:
low: {reasoning: {effort: low, summary: auto}}
medium: {reasoning: {effort: medium, summary: auto}}
high: {reasoning: {effort: high, summary: auto}}
```

```python
from pathlib import Path
from nooa.unifiedllm import get_llm_client
from nooa.unifiedllm.registry import reload_registry

reload_registry(Path("llm_config.yaml"))
llm = get_llm_client("my-route", reasoning_level="high") # persistent selection
levels = llm.reasoning_levels
default = llm.reasoning_default
# Pass llm to an Agent through the normal cascade below.
# For a direct UnifiedLLM call, override only this request:
reply = await llm.acall(messages, reasoning_level="low")
```

- Omit `reasoning_levels` (or use null) for unknown support; `{}` declares
selection unsupported. The public property returns `None`, `()`, or a tuple
of valid labels respectively. Invalid selections raise, naming the choices.
- `reasoning_default` is metadata, not a selection: if supplied it must name
a declared level and describe the base configuration. `reasoning_level=None`
on a call bypasses a persistent selection and uses those base parameters.
- Declare levels/defaults on the registry entry or client constructor, never
per-call or in `extra_body`. Do not combine a selected level with a per-call
raw setting of the same key (including inside `extra_body`); that raises.
Level blocks cannot change routing, credentials, messages or framework controls.
Construct a new client when changing routes rather than reusing its level map.
- Use provider documentation or catalogues as candidates, then check the actual
route's outgoing HTTP fields: SDKs and gateways may reject or silently drop
settings. HTTP acceptance alone does not prove a level was honored. Leave
unverified choices unknown rather than inventing mappings.
- This controls effort, not retention of stored reasoning. Changing effort may
invalidate the cached prefix; labels are not comparable across providers.

See [reasoning-level configuration](../../docs/reasoning-levels.md) for reserved
keys, transport caveats, and bounded validation commands. The
`examples/reasoning_levels/llm_config.yaml` entries require explicit loading;
they are not automatically available registry aliases.

**Resolution cascade** for which LLM a method uses — first match wins:

1. `await agent.method(..., llm=special_llm)` — call override
Expand Down
3 changes: 3 additions & 0 deletions src/nooa/config/model_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,9 @@ class ModelConfig(BaseModel):
max_tokens: int | None = None
temperature: float | None = None
top_p: float | None = None
reasoning_levels: dict[str, dict[str, Any]] | None = None
reasoning_default: str | None = None
reasoning_level: str | None = None

@classmethod
def from_registry(cls, name: str, raw: dict[str, Any]) -> ModelConfig:
Expand Down
2 changes: 2 additions & 0 deletions src/nooa/unifiedllm/fake.py
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ async def acall(
Thread-safe: uses asyncio.Lock to ensure concurrent calls get responses in order.
"""
async with self._lock:
self._prepare_call_config(kwargs)
self.call_count += 1
# A non-provider test client must never observe private replay state.
self.last_messages, _, _ = apply_cache_policy(
Expand Down Expand Up @@ -114,6 +115,7 @@ def call(
) -> LLMResponse:
"""Synchronous version of acall for UnifiedLLM compatibility."""
# For sync call, we don't need locking since tests are usually single-threaded
self._prepare_call_config(kwargs)
self.call_count += 1
self.last_messages, _, _ = apply_cache_policy(
prepare_chat_messages(messages, None), None, responses=False
Expand Down
120 changes: 120 additions & 0 deletions src/nooa/unifiedllm/reasoning.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
"""Declared effort choices, independent of model names and provider discovery."""

from collections.abc import Mapping
from copy import deepcopy
from typing import Any

from pydantic import BaseModel, ConfigDict, model_validator

_DECLARATIONS = {"reasoning_levels", "reasoning_default"}
# These select the client/request itself, not a provider's effort behavior.
_RESERVED = _DECLARATIONS | {
"reasoning_level",
"model",
"api_base",
"base_url",
"api_key",
"custom_llm_provider",
"messages",
"input",
"extra_body",
"client",
}


class ReasoningConfig(BaseModel):
"""Map public level names to request settings for one configured route.

None means unknown support; an empty mapping means unsupported. The default
documents the route's default, not a request to send it on every call.
Declarations live in registry YAML, not a model-name table in this module.
"""

model_config = ConfigDict(extra="forbid", frozen=True)

levels: dict[str, dict[str, Any]] | None = None
Comment thread
coderabbitai[bot] marked this conversation as resolved.
default: str | None = None

@model_validator(mode="after")
def validate_declaration(self):
"""Reject malformed choices and request-control fields at construction."""
if self.default is not None and self.default not in (self.levels or {}):
raise ValueError("reasoning_default must name a declared reasoning level")
for level, settings in (self.levels or {}).items():
if not level.strip() or not settings:
raise ValueError("reasoning_levels must have non-empty names and request settings")
Comment thread
furgalep marked this conversation as resolved.
if conflict := _RESERVED & settings.keys():
raise ValueError(
f"reasoning level {level!r} contains reserved fields: {sorted(conflict)}"
)
return self

def settings(self, level: str) -> dict[str, Any]:
"""Validate a selection and detach its settings from the stored declaration."""
if self.levels is None:
raise ValueError(
"Reasoning levels are unknown for this route; declare reasoning_levels"
)
if not self.levels:
raise ValueError("Reasoning-level selection is not supported for this route")
if not isinstance(level, str) or level not in self.levels:
raise ValueError(
f"Invalid reasoning level {level!r}; allowed: {', '.join(self.levels)}"
)
# Frozen Pydantic attributes do not freeze nested dictionaries. Reuse the
# declaration checks so later edits cannot introduce routing controls.
self.validate_declaration()
# Only the small chosen configuration is copied, never conversation data.
return deepcopy(self.levels[level])


def apply_reasoning_level(
declaration: ReasoningConfig,
model: str,
defaults: dict[str, Any],
overrides: dict[str, Any],
default_selection: str | None,
) -> dict[str, Any]:
"""Resolve effort once before either client dispatches.

No selection leaves existing provider settings untouched. An explicit level
replaces constructor defaults; mixing it with per-call native controls is an
error. Route changes cannot inherit a declaration for a different endpoint.
This affects requested effort, never stored reasoning or replay compatibility.
"""
if _DECLARATIONS & overrides.keys():
raise ValueError("reasoning_levels and reasoning_default belong on the client constructor")
params = {**defaults, **overrides}
extra = params.get("extra_body")
if isinstance(extra, Mapping) and (set(extra) & (_DECLARATIONS | {"reasoning_level"})):
raise ValueError("Reasoning configuration cannot be passed through extra_body")
level = params.pop("reasoning_level", default_selection)
if level is None:
return params
patch = declaration.settings(level)
if (
any(
key in overrides and overrides[key] != defaults.get(key)
for key in ("api_base", "base_url", "custom_llm_provider")
Comment thread
furgalep marked this conversation as resolved.
)
or overrides.get("model", model) != model
or overrides.get("client") is not None
):
raise ValueError("Reasoning levels are route-specific; create a client for the new route")
explicit_extra = overrides.get("extra_body")
if conflict := patch.keys() & (
overrides.keys() | (explicit_extra.keys() if isinstance(explicit_extra, Mapping) else set())
):
raise ValueError(
f"reasoning_level conflicts with explicit request field(s): {sorted(conflict)}"
)
# Whole top-level values replace defaults. Authors write complete nested
# blocks in YAML; no provider-specific merge or inheritance rules live here.
# Remove replaced defaults from extra_body too: SDKs otherwise merge those
# back over the selected top-level values when assembling the HTTP body.
if isinstance(extra, Mapping) and patch.keys() & extra.keys():
params["extra_body"] = {key: value for key, value in extra.items() if key not in patch}
params.update(patch)
return params
Loading
Loading