Problem
Every Anthropic provider child retrieves the selected model's output limit before it can start generation. That metadata is not cached across user turns. If a different compaction model is configured, its metadata is also fetched immediately, even when the turn never needs compaction.
This introduces one or two extra Models API calls on each user request, increases startup latency, and makes ordinary generation depend on an unused compaction-model lookup succeeding. These are API/latency costs, not generation-token charges.
Evidence and reproduction
Audited main at 7289751459ceace471ac771f3a0ec39b972d1949. Area: AI provider orchestration. Findings come from source inspection and isolated Python execution of the relevant source functions with mocked CAD/model boundaries; no paid model requests or live CAD mutations were made.
Executing the actual lookup helper ten times with the same fake client/model produced ten metadata retrievals. Source tracing confirms startup invokes it on each new child. With distinct primary/compaction models, the current startup path performs two lookups even for a successful one-response turn.
Proposed fix
Add a bounded capability cache in the parent/provider lifetime and pass validated capability metadata to the child. Scope entries to endpoint, model, and authentication context; use a TTL or explicit refresh so aliases and capabilities can change. Do not store credentials in cache telemetry.
Resolve a different compaction model lazily when compaction is actually needed. Preserve the model-reported output limits and SDK compatibility behavior introduced deliberately in #153; do not replace them with hard-coded small output caps.
Acceptance tests
- Repeated turns using the same endpoint/model reuse fresh metadata across child processes.
- Model, endpoint, authentication changes, and expiry trigger appropriate refresh.
- No compaction-model lookup occurs on a turn that does not compact.
- A compaction turn resolves the correct model's capabilities.
- Invalid/failed metadata is not cached as a valid capability.
- Existing maximum-output and older-SDK compatibility tests continue to pass.
Suggested priority: low; a small independent API-efficiency PR.
Problem
Every Anthropic provider child retrieves the selected model's output limit before it can start generation. That metadata is not cached across user turns. If a different compaction model is configured, its metadata is also fetched immediately, even when the turn never needs compaction.
This introduces one or two extra Models API calls on each user request, increases startup latency, and makes ordinary generation depend on an unused compaction-model lookup succeeding. These are API/latency costs, not generation-token charges.
Evidence and reproduction
Audited
mainat7289751459ceace471ac771f3a0ec39b972d1949. Area: AI provider orchestration. Findings come from source inspection and isolated Python execution of the relevant source functions with mocked CAD/model boundaries; no paid model requests or live CAD mutations were made.models.retrieve(model)every time.Executing the actual lookup helper ten times with the same fake client/model produced ten metadata retrievals. Source tracing confirms startup invokes it on each new child. With distinct primary/compaction models, the current startup path performs two lookups even for a successful one-response turn.
Proposed fix
Add a bounded capability cache in the parent/provider lifetime and pass validated capability metadata to the child. Scope entries to endpoint, model, and authentication context; use a TTL or explicit refresh so aliases and capabilities can change. Do not store credentials in cache telemetry.
Resolve a different compaction model lazily when compaction is actually needed. Preserve the model-reported output limits and SDK compatibility behavior introduced deliberately in #153; do not replace them with hard-coded small output caps.
Acceptance tests
Suggested priority: low; a small independent API-efficiency PR.