Skip to content

[codex] Add LLM capability snapshot dispatch#955

Draft
mimeding wants to merge 1 commit into
osaurus-ai:mainfrom
mimeding:codex/llm-capability-snapshot
Draft

[codex] Add LLM capability snapshot dispatch#955
mimeding wants to merge 1 commit into
osaurus-ai:mainfrom
mimeding:codex/llm-capability-snapshot

Conversation

@mimeding
Copy link
Copy Markdown
Contributor

@mimeding mimeding commented Apr 26, 2026

Business rationale

Model routing, prompt budgeting, tool availability, and provider request behavior are part of the Osaurus harness: users benefit when a model's real capabilities are explicit instead of inferred by scattered string checks. This snapshot gives local MLX, Apple Foundation Models, OpenAI-compatible providers, Azure OpenAI, Anthropic, Gemini, Codex, and Osaurus providers one inspectable contract for runtime kind, modalities, reasoning, tool use, streaming, and context behavior.

Coding rationale

The capability value lives in configuration models so it can be shared by routing, request construction, UI gating, and tests without pulling service state into view code. ChatEngine resolves the snapshot after the effective service/model pair is known, which avoids guessing before fallback routing has settled. ModelService receives the snapshot through GenerationParameters, keeping later adapter pruning and diagnostics on the same path for streaming and non-streaming calls. Tests cover provider mapping, model-profile hints, Azure OpenAI compatibility, and dispatch propagation so future provider additions have a clear contract to extend.

What changed

  • Added LLMCapabilitySnapshot, provider/runtime enums, modality flags, and resolver logic.
  • Threaded the snapshot through ChatEngine dispatch into GenerationParameters.
  • Added snapshot-focused tests for local, Foundation, OpenAI-compatible, Azure OpenAI, Gemini, and capability-profile behavior.
  • Added ChatEngine tests proving both streaming and non-streaming dispatch preserve the resolved snapshot.
  • Cleaned the rebased branch back to the capability snapshot scope, dropping unrelated stale test-gate and TTS history now covered elsewhere.

Validation

  • swift build --package-path Packages/OsaurusCore
  • swift build --package-path Packages/OsaurusCore -c release
  • swift test --package-path Packages/OsaurusCore — 1447 tests in 193 suites passed.
  • xcrun swift-format lint --strict Packages/OsaurusCore/Models/Configuration/LLMCapabilitySnapshot.swift Packages/OsaurusCore/Services/Chat/ChatEngine.swift Packages/OsaurusCore/Services/Inference/ModelService.swift Packages/OsaurusCore/Tests/Chat/ChatEngineTests.swift Packages/OsaurusCore/Tests/Model/LLMCapabilitySnapshotTests.swift
  • swiftlint lint --strict Packages/OsaurusCore/Models/Configuration/LLMCapabilitySnapshot.swift Packages/OsaurusCore/Services/Chat/ChatEngine.swift Packages/OsaurusCore/Services/Inference/ModelService.swift Packages/OsaurusCore/Tests/Chat/ChatEngineTests.swift Packages/OsaurusCore/Tests/Model/LLMCapabilitySnapshotTests.swift
  • git diff --check

Non-scope

  • Does not prune provider-specific request parameters yet.
  • Does not change visible provider settings or model-picker UI gating.
  • Does not alter prompt-pack selection beyond passing the capability contract forward.

Residual risks

  • Some third-party provider model names will still need profile metadata to describe capabilities precisely.
  • The snapshot is intentionally conservative; later PRs should expand it only when a user-facing behavior consumes the new field.

@mimeding mimeding force-pushed the codex/llm-capability-snapshot branch from 9791311 to 270e20f Compare April 27, 2026 12:32
@mimeding mimeding force-pushed the codex/llm-capability-snapshot branch 5 times, most recently from 798fc7f to 6d294ba Compare April 29, 2026 18:58
@mimeding mimeding marked this pull request as ready for review April 29, 2026 18:58
@mimeding mimeding force-pushed the codex/llm-capability-snapshot branch 3 times, most recently from ce1cac4 to c8e6dec Compare May 1, 2026 18:11
Business rationale: model routing and prompt/tool decisions are part of the Osaurus harness, and users need them to stay consistent across local, Foundation, and remote providers without each call site guessing differently.

Coding rationale: the snapshot centralizes provider, runtime, modality, reasoning, tool, and context traits in one value type; ChatEngine and ModelService consume that contract instead of ad hoc checks, while tests pin both service dispatch and provider mapping including Azure OpenAI.

Co-authored-by: Michael Meding <mmeding@Michaels-Mac-Studio.local>
@mimeding mimeding force-pushed the codex/llm-capability-snapshot branch from c8e6dec to 09fff95 Compare May 3, 2026 21:36
@mimeding mimeding marked this pull request as draft May 10, 2026 14:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant