Skip to content

Conversation

@vojto
Copy link

@vojto vojto commented Dec 5, 2025

Hey, I tried to support structured output for Anthropic.

The only problem is that RubyLLM::Schema currently adds strict: true to each schema it creates. Here's a PR in that repo to fix that: danielfriis/ruby_llm-schema#28

Until that is merged, the workaround is to create this subclass of RubyLLM::Schema:

class AnthropicSchema < RubyLLM::Schema
  def to_json_schema
    result = super
    result[:schema].delete(:strict)
    result[:schema].delete("strict")
    result
  end
end

AI generated:

Summary

  • Implements structured outputs using Anthropic's structured-outputs-2025-11-13 beta API
  • Adds structured output capability detection for Claude 4+ models
  • Includes comprehensive test coverage for the new functionality

Changes

  • Anthropic Provider: Added complete method override to inject the structured-outputs beta header when schema is provided
  • Capabilities: Added supports_structured_output? method to detect Claude 4+ models that support structured outputs
  • Chat: Implemented output_format parameter with json_schema type in payload rendering
  • Tests: Added specs for beta header handling and output_format payload generation
  • Refactoring: Extracted claude3_or_newer? and claude4_or_newer? helper methods for cleaner version detection

Test plan

  • Added unit tests for beta header handling
  • Added unit tests for output_format payload generation
  • Verified structured output capability detection for Claude 4+ models
  • Manual testing with actual Anthropic API (recommended)

🤖 Generated with Claude Code

Implements structured outputs using Anthropic's structured-outputs-2025-11-13 beta API.

Changes:
- Add structured output capability detection for Claude 4+ models
- Implement output_format parameter with json_schema type in chat payload
- Add anthropic-beta header handling to append structured-outputs beta version
- Add comprehensive specs for structured output functionality
- Refactor model version detection helpers (claude3_or_newer, claude4_or_newer)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@vojto vojto marked this pull request as draft December 5, 2025 08:00
vojto and others added 2 commits December 5, 2025 09:37
Move schema validation logic into dedicated class to reduce complexity
in the Chat module and improve separation of concerns.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
The spec file legitimately tests two separate classes (Provider and Chat module) that are closely related but have distinct responsibilities. Disabling this cop for this file is the appropriate solution.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@tpaulshippy
Copy link
Contributor

Do you plan to add or modify specs that hit the real API and produce new/updated VCR cassettes?

@vojto
Copy link
Author

vojto commented Dec 8, 2025

@tpaulshippy that should be done now.

I edited models.json manually, because I don't have all the API keys to run rake models:update. I'm assuming someone else will update it after merging the PR?

I modified models_to_test.rb a little - made a separate array for models that we wanna test for schema. I added haiku-4.5 and sonnet-4.5 to this list, and kept gpt-4.1-nano and gemini-2.5-flash that was already there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants