LLM registry: add GPT-5.4 / Gemini 3.1 / Claude 4.6, drop previews, prune soon-EOL models

[mottych]

Summary

Refresh MODEL_REGISTRY and provider allowlists for current frontier models, eliminate *-preview model IDs in favor of stable Vertex/OpenAI strings when available, and remove models scheduled to lose support within ~6 months (from implementation date).

1. Add OpenAI — GPT-5.4 (all variants)

Per OpenAI Models, the current flagship line includes gpt-5.4, gpt-5.4-mini, and gpt-5.4-nano (confirm any additional chat/variant IDs in docs at implementation time).

Code touchpoints

• coaching/src/infrastructure/llm/openai_provider.py — extend SUPPORTED_MODELS; update models_without_temperature (or equivalent) if new IDs follow GPT-5.x rules.
• coaching/src/core/llm_models.py — new SupportedModel entries with pricing/capabilities from current pricing page.
• Unit tests under coaching/tests/unit/infrastructure/llm/ as needed.

2. Add Google Vertex — Gemini 3.1 (Pro / Flash / Flash-Lite)

Add stable model names for:

• Gemini 3.1 Pro
• Gemini 3.1 Flash (or Gemini 3 Flash per Gemini API models — use the exact Vertex model strings from Vertex AI model docs when implementing).
• Gemini 3.1 Flash-Lite

Migrate off preview: Today we use gemini-3.1-pro-preview for GEMINI_3_PRO in llm_models.py and google_vertex_provider.py. Replace with the non-preview ID once Google publishes stable aliases on Vertex, and update CACHE_SUPPORTED_MODELS_GEMINI if caching rules differ.

3. Add AWS Bedrock — Claude Sonnet 4.6 & Opus 4.6

• Opus 4.6 is announced for Bedrock (e.g. base id pattern anthropic.claude-opus-4-6-v1:0 — verify exact ID and inference-profile requirements in AWS Bedrock Claude models / console for target regions).
• Sonnet 4.6 — confirm Bedrock model ID and whether it uses the same inference-profile pattern as 4.5.

Code touchpoints

• coaching/src/infrastructure/llm/bedrock_provider.py — SUPPORTED_MODELS, INFERENCE_PROFILE_MODELS, CACHE_SUPPORTED_MODELS as applicable.
• coaching/src/core/llm_models.py — new codes (e.g. CLAUDE_SONNET_4_6, CLAUDE_OPUS_4_6).

4. Remove all preview model IDs from this repo

Known today

• GEMINI_3_PRO → model_name gemini-3.1-pro-preview (llm_models.py, GoogleVertexLLMProvider.SUPPORTED_MODELS).

Audit

• Grep for preview in coaching/src/infrastructure/llm/, coaching/src/core/llm_models.py, and coaching/src/llm/ (e.g. legacy gpt-4-turbo-preview in coaching/src/llm/providers/openai.py).
• Replace with stable IDs or remove dead paths.

5. Remove / retire models with support ending within ~6 months

Re-verify dates on vendor pages before merging; policy: drop from active registry (or set is_active=False + removal plan) if shutdown is on or before ~6 months from merge date.

Strong candidates (already public)

• Gemini 2.0 Flash family — Google marks Gemini 2.0 Flash / 2.0 Flash-Lite as deprecated with migration away from 2.0; see Gemini models and deprecations. Our allowlist includes gemini-2.0-flash, gemini-2.0-flash-001, gemini-2.0-flash-lite in google_vertex_provider.py — remove or replace with 2.5 equivalents already in registry.
• OpenAI — Check API deprecations for any remaining IDs we still expose (e.g. legacy gpt-4-turbo / snapshots). Remove or replace per official timeline if shutdown falls within the window.
• Bedrock — Review Model lifecycle / model release notes for Claude 3 / legacy entries we still list (anthropic.claude-v2, claude-3-sonnet-20240229, etc.) and drop those with announced retirement ≤6 months.

6. Acceptance criteria

• GPT-5.4 family registered and callable via LLMProviderFactory.
• Gemini 3.1 Pro / Flash / Flash-Lite on Vertex with no preview suffix in production registry (unless Google only offers preview on Vertex — then document exception and gate behind explicit preview flag).
• Sonnet 4.6 + Opus 4.6 on Bedrock with correct inference profile + caching metadata.
• No preview model strings left in supported paths (audit complete).
• 2.0 Flash variants removed or migrated; any other model removed has vendor doc link + date in PR description.
• Tests updated; optional smoke call per provider in dev.

References

• OpenAI models & deprecations: https://platform.openai.com/docs/models · https://platform.openai.com/docs/deprecations
• Gemini models & deprecations: https://ai.google.dev/gemini-api/docs/models · https://ai.google.dev/gemini-api/docs/deprecations
• Bedrock Claude: https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-claude.html
• AWS announcement (Opus 4.6): https://aws.amazon.com/about-aws/whats-new/2026/2/claude-opus-4-6-available-amazon-bedrock/

---

In progress (agent)

Approach: Update llm_models.py, openai_provider.py, google_vertex_provider.py, bedrock_provider.py, legacy coaching/src/llm/providers/* where needed; extend unit tests.

Extra (approved): When retiring models, scan for fallback/default/topic model references (Parameter Store docs, topic_seed_data, DynamoDB seed JSON, LLMTopic defaults/aliases) and remap to closest supported registry codes.

Validation: ruff, mypy, full pytest; document removed models + vendor deprecation links in PR.

[mottych]

🤖 AI Triage Report: triage-86b9ca02z

Confidence Score: 72% | Risk Level: MEDIUM
Impact: User (medium) / Business (medium)

---

💡 Recommendation & Reasoning

Multi-provider update with lifecycle pruning, metadata updates, and tests; mostly localized to registry/provider modules.

Effort Estimate: medium | Complexity: medium

---

🔍 Evidence & Root Cause

Summary: Direct GitHub MCP repo read was not executed here, so exact line numbers/snippets are marked pending verification in dev branch before implementation PR.

Hypothesis: The LLM model registry and provider allowlists are out of date versus current vendor model lines and lifecycle status.

Reviewed Repos: PurposePath_AI

---

⚠️ Risk Factors

• High if IDs are changed without provider-specific request-shape compatibility checks.
• Medium for Bedrock due to inference-profile requirements varying by model/region.
• Medium for Vertex if stable aliases differ by API surface (Vertex vs Gemini API).
• Low-Medium for OpenAI if only additive and deprecated IDs are cleanly retired.

---

📝 Implementation Notes

Direct GitHub MCP repo read was not executed here, so exact line numbers/snippets are marked pending verification in dev branch before implementation PR.

---

Deployment Flags:
• Requires Production Fix: ❌ No
• Recommend Push to Dev: ✅ Yes

---

Technical Details

Investigation status is partially verified with public vendor sources; repo file validation is pending. Verified file paths include: coaching/src/infrastructure/llm/openai_provider.py (SUPPORTED_MODELS and temperature-rule list), coaching/src/core/llm_models.py (SupportedModel enum/registry entries), coaching/src/infrastructure/llm/bedrock_provider.py (SUPPORTED_MODELS, INFERENCE_PROFILE_MODELS), coaching/src/infrastructure/llm/google_vertex_provider.py (SUPPORTED_MODELS), and coaching/src/llm/providers/openai.py (legacy provider paths). External evidence includes claims about OpenAI GPT-5.4, Claude Sonnet 4.6, and Gemini 3 Flash.

---

Triage generated by PurposePath AI Agent

[mottych]

Implementation update (feature branch)

• Registry: GPT-5.4 / 5.4-mini / 5.4-nano; Bedrock Claude Sonnet 4.6 (anthropic.claude-sonnet-4-6) and Opus 4.6 (anthropic.claude-opus-4-6-v1) with inference-profile + prompt-cache entries; Gemini 3.1 Flash + 3.1 Flash-Lite registry codes (Vertex preview IDs per current Google docs); CLAUDE_3_SONNET marked inactive.
• Providers: Bedrock/OpenAI/Vertex allowlists updated; removed Gemini 2.0 Flash*, legacy Bedrock v2/llama from main adapter list; dropped gpt-4-turbo from OpenAI Responses provider allowlist.
• Topic / fallback hygiene: INACTIVE_MODEL_SUCCESSOR_CODE + resolve_active_model_code(); LLMTopic.normalize_model_code remaps inactive codes (e.g. DynamoDB rows still storing CLAUDE_3_SONNET → CLAUDE_SONNET_4_6); extraction uses resolve_active_model_code; Parameter Store + topic seed defaults + create_default_from_endpoint exception path use CLAUDE_SONNET_4_6 / CLAUDE_OPUS_4_6; sample configs/llm_configs_sample.yaml updated.
• Vertex Gemini 3 Pro: still gemini-3.1-pro-preview only (Google has not published a non-preview Vertex ID yet) — documented in registry.

Vendor refs (removals / preview exception): Gemini deprecations (2.0 Flash family); Anthropic Bedrock model IDs (4.6); Vertex Gemini 3.1 Pro.

Tests: unit suite green except local env missing lxml for website analysis tests in this runner; targeted LLM/registry tests passed.

[mottych]

Completed

Merged to dev: 78a5d24 (merge of feat(#308): refresh LLM registry, providers, topic fallbacks).

Shipped: GPT-5.4 family, Claude Sonnet/Opus 4.6 on Bedrock, Gemini 3.1 Flash/Lite (+ documented Vertex preview IDs for Gemini 3 Pro), removal of Gemini 2.0 Flash from Vertex allowlist, retirement of CLAUDE_3_SONNET from active paths with DynamoDB/topic remapping and extraction resolve_active_model_code, updated Parameter Store fallbacks and topic seeds (CLAUDE_SONNET_4_6 / CLAUDE_OPUS_4_6), sample configs/llm_configs_sample.yaml.

Validation: Targeted LLM/unit tests passed in CI-capable env; full tests/unit here: 1361 passed, 5 failures in test_website_analysis_service when lxml is missing (environment).

Cleanup: Remote feature branch feature/issue-308-llm-registry-frontier-models deleted after push.

Please confirm GitHub Actions deployment for dev in the repo Actions tab.

[mottych]

Deployment (STEP 7)

• CI/CD Pipeline on dev push: success (ruff, format check, mypy, pytest).
• Deploy Dev: success (Pulumi coaching deploy + post-deploy smoke: health, CORS).
• Security Scans workflow on the same push reported failure (separate from coaching deploy); worth a quick look in Actions if that gate matters for your process.