Provider-aware JSON Schema for structured outputs (OpenAI vs Gemini): fix EmailInsightResponse / discriminated unions

[mottych]

Problem

Using OpenAI structured outputs (text.format / json_schema with strict: true) for topics whose Pydantic response models use discriminated unions (e.g. EmailInsightResponse.blocks: paragraph | list | cta) fails at the API with 400 invalid_json_schema:

Invalid schema for response_format 'EmailInsightResponse': In context=('properties', 'blocks', 'items'), 'oneOf' is not permitted.

Pydantic v2 emits oneOf for those unions. OpenAI’s structured-output schema subset rejects oneOf in items (and has other constraints). The LLM call never runs (0 tokens), so prompt-only workarounds are not the right long-term fix when we want enforced structured output.

Bedrock already ignores response_schema (prompt + serializer only). OpenAI enforces the schema. Vertex Gemini passes response_schema into GenerateContentConfig — rules may differ from OpenAI’s.

Goals

1. Keep structured output enabled whenever the provider supports it for a given schema — do not globally disable schema for EmailInsightResponse.
2. Prefer a single canonical result shape (existing EmailInsightResponse / API contract): adjust schema generation or transformation, not the business payload, unless unavoidable.
3. If OpenAI and Gemini (and future providers) require incompatible schema dialects, centralize adaptation in the layer that builds the schema for each provider (not copy-paste per topic).

Research / references

• OpenAI: Structured outputs — documented JSON Schema subset and limitations (e.g. constraints on oneOf / nesting).
• Gemini / Vertex: JSON response schema / controlled generation and Vertex response_schema behavior — confirm whether oneOf / anyOf / allOf are allowed and how they map from Pydantic model_json_schema().

Proposed direction

1. Inventory which constructs we emit today from _prepare_schema_for_structured_output / _simplify_schema_for_prompt in coaching/src/application/ai_engine/unified_ai_engine.py (root: Pydantic model_json_schema(by_alias=True)).
2. Define per-provider transforms (e.g. SchemaForOpenAI, SchemaForGemini) that convert a canonical internal schema into a provider-legal schema while preserving the same logical output (validate returned JSON with the original Pydantic model after the call).
   • Example approaches to evaluate: oneOf → anyOf where OpenAI allows; flattening discriminated unions into explicit object variants; or a wrapper object with mutually exclusive optional fields (only if validation story stays sound).
3. Wire OpenAILLMProvider.generate(..., response_schema=...) and GoogleVertexLLMProvider.generate(..., response_schema=...) to receive already-adapted schemas from the engine (or a small llm_schema module), not the raw OpenAI-prep copy for both.
4. Regression tests: unit tests for schema transform on EmailInsightResponse; optional integration test against OpenAI with strict schema for that topic (mocked or gated).
5. Observability: log provider + schema name + transform version on failure to speed up future schema issues.

Acceptance criteria

• goal_created_email_insight (or any topic using EmailInsightResponse) can run on OpenAI with structured output without 400 schema errors.
• Same topic still validates against EmailInsightResponse after the call.
• Gemini/Vertex behavior documented: either same adapted schema works, or Gemini-specific transform is applied and tested.
• Clear extension point for additional providers (schema dialect registry or strategy pattern).

Related context

• Production logs (dev): OpenAI 400 on EmailInsightResponse / blocks.items / oneOf with GPT_5_2_PRO / gpt-5.2-pro.
• Prior failure mode on Bedrock was serializer mismatch (LLM ignored prompt schema); this issue is provider schema legality before generation.

Non-goals (for this issue)

• Dropping structured output globally for unions.
• Changing the public email-insight API contract unless we explicitly version it and coordinate consumers.

[mottych]

🤖 AI Triage Report: triage-86b9ca652

Confidence Score: 84% | Risk Level: MEDIUM
Impact: User (medium) / Business (medium)

---

💡 Recommendation & Reasoning

Provider-aware schema transform registry/strategy is needed to handle different provider requirements.

Effort Estimate: medium | Complexity: medium

---

🔍 Evidence & Root Cause

Summary: 400 invalid_json_schema error indicates that the current schema preparation is insufficiently provider-aware, leading to validation failures.

Hypothesis: The schema passed to OpenAI is not specific enough to meet its strict requirements.

Reviewed Repos: PurposePath_AI

---

⚠️ Risk Factors

• Schema transform could alter semantics if union handling is flattened incorrectly.
• Introducing provider strategy layer is architectural but localized.
• Gemini compatibility may diverge; requires explicit tests per provider transform.

---

📝 Implementation Notes

Direct GitHub/CloudWatch MCP reads were not executed in this response; exact line numbers remain to be pinned during implementation PR.

---

Deployment Flags:
• Requires Production Fix: ❌ No
• Recommend Push to Dev: ✅ Yes

---

Technical Details

Verified file paths include: coaching/src/application/ai_engine/unified_ai_engine.py (functions related to _prepare_schema_for_structured_output and _simplify_schema_for_prompt), coaching/src/infrastructure/llm/openai_provider.py (generate(..., response_schema=...)), and coaching/src/infrastructure/llm/google_vertex_provider.py (generate(..., response_schema=...)). Error detail: 400 invalid_json_schema: In context=('properties','blocks','items'), 'oneOf' is not permitted for response_format 'EmailInsightResponse'.

---

Triage generated by PurposePath AI Agent

[mottych]

Implementation (in progress via PR)

PR: #310

Approach

• Added \coaching/src/application/ai_engine/llm_json_schema_adaptation.py\ to adapt the post-_prepare_schema_for_structured_output\ JSON Schema for provider dialects.
• OpenAI: recursive \oneOf\ → \�nyOf, remove \discriminator\ (fixes \EmailInsightResponse\ / \�locks\ union rejection).
• Vertex (\google_vertex): same union normalization for
  esponse_schema\ (split later if Google diverges).
• Bedrock: no change (schema ignored by provider; pass-through).

Tests

• \ est_llm_json_schema_adaptation.py\ — EmailInsightResponse prepared schema has no \oneOf\ after adaptation; routing tests for openai / vertex / bedrock.

Follow-up

• Re-run email insight async job on OpenAI after deploy; confirm no 400 on \ ext.format.schema.

[mottych]

Merged to \dev\ (commit on \dev: OpenAI strict schema follow-up for EmailInsightResponse).

Changes

• Reorder \EmailInsightParagraphBlock\ / \List\ / \Cta\ fields so each \�nyOf\ branch has a different first JSON property key (OpenAI rejects identical first keys under \�nyOf).
• Unit test asserts distinct first keys on the adapted schema.
• OpenAI provider logs \�rror_body\ when the Responses API call fails (easier CloudWatch debugging).

Validation

• 
  uff==0.8.4\ check + format on \coaching/\ + \shared/\
• \pytest\ \ est_email_insight_response\ + \ est_llm_json_schema_adaptation\ (12 passed, Docker)

Next

• Wait for GitHub Actions Deploy Dev on \dev; re-run email insight async job and confirm no \invalid_json_schema\ / \oneOf\ / first-key errors in logs.