Skip to content

Fix Pi routing and broaden live deployment validation - #325

Merged
16francej merged 5 commits into
mainfrom
codex/fix-aux-model-routing
Aug 11, 2026
Merged

Fix Pi routing and broaden live deployment validation#325
16francej merged 5 commits into
mainfrom
codex/fix-aux-model-routing

Conversation

@16francej

@16francej 16francej commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Summary

  • derive Pi's default and auxiliary models from the selected provider when no explicit model is pinned, and surface auxiliary failures through the session error log
  • make qm check --live run a private end-to-end canary that requires the exact model reply, durable transcript, generated title, and an empty synchronized error log
  • keep the canary out of administrator memory and archive its session on both success and failure
  • execute the canary over Fly loopback or as an AWS one-off task on the core service's private network, with stack-scoped IAM permissions and no new public endpoint
  • update the deployment skill and handoff guidance around the broader gate, published as CLI 0.1.6

Root cause

Provider-only deployments selected the correct fallback for agent turns in the harness router, but Pi's model utilities did not receive that fallback. An OpenAI deployment could therefore serve turns with gpt-5.6-sol while title generation independently fell back to an Anthropic model; the missing credential was then treated like a legitimate no-title result.

The deployment workflow also stopped at infrastructure, routing, storage, and public-health checks. It never exercised the full model-to-session path, so a deployment could pass automation without proving model execution, transcript persistence, title generation, or session error reporting.

Deployment impact

qm check --live now runs one real agent turn plus auxiliary title generation after the existing provider checks pass. Fly executes it over loopback inside core. AWS launches only the deployed core task definition on the same private VPC and reaches core through service discovery. The turn is read-only, skips memory recall and capture, and is always archived.

Existing AWS deployments must render, review, and apply the updated Terraform policy with infrastructure-administrator credentials before enabling this gate. Administrator sign-in, a manual web acceptance turn, agent-computer proof, and Slack acceptance when configured remain manual checks.

Verification

  • failing-before/passing-after regressions for provider-only Pi routing, auxiliary title errors, memory isolation, exact replies, failed-session cleanup, synchronized Postgres event reads, and scoped AWS canary permissions
  • focused postdeploy, orchestrator, source-auth, Fly, AWS, Terraform, and deployment-skill tests
  • root and CLI TypeScript checks; ESLint, Oxlint, and Prettier
  • Terraform formatting and validation against the production AWS deployment configuration
  • real private Pi/OpenAI canary against an isolated production-shaped stack: exact reply, persistence, generated title, synchronized error-log check, and archival all passed

No rendered UI code changed, so no screenshot is required.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

@16francej 16francej changed the title Fix Pi auxiliary model routing and deploy verification Fix Pi routing and broaden live deployment validation Aug 11, 2026
@16francej
16francej marked this pull request as ready for review August 11, 2026 21:48
@16francej
16francej merged commit 3cb5623 into main Aug 11, 2026
20 checks passed
@16francej
16francej deleted the codex/fix-aux-model-routing branch August 11, 2026 21:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant