fix: Responses API native content types in adapter

[callingmedic911]

Summary

• OpenAIResponsesAdapter now produces Responses API native content types (input_text, input_image, input_file) instead of inheriting Chat Completions types (text, image_url, file) from OpenAILLMAdapter
• OpenAI rejects Chat Completions content block types when sent via the Responses API input parameter
• Bumps version to 0.6.1

Note

Fix OpenAIResponsesAdapter to emit Responses API-native content block types

• Rewrites OpenAIResponsesAdapter in src/model.ts to implement ILLMAdapter directly instead of extending OpenAILLMAdapter, so content blocks are no longer converted to Chat Completions types.
• Text content maps to {type: "input_text"}, image URLs to {type: "input_image"}, base64 images to {type: "input_image"} with a data URI, and base64 files to {type: "input_file"} with a derived filename.
• Adds a private formatBase64Content helper to handle base64 image and file cases.
• Risk: non-image media URLs and base64 audio now throw errors at formatting time rather than passing through silently.

^{Macroscope summarized fd1a2c7.}

Summary by CodeRabbit

Release Notes v0.6.1

• Bug Fixes

  • Fixed OpenAI Responses API adapter to use native content types (input_text, input_image, input_file) instead of Chat Completions types, resolving compatibility issues with OpenAI's API.

• Tests

  • Added comprehensive test suite for content block type conversion and error handling in the OpenAI Responses adapter.

[coderabbitai]

📝 Walkthrough

Walkthrough

The PR updates the OpenAIResponsesAdapter to use OpenAI Responses API native content types (input_text, input_image, input_file) instead of Chat Completions types. Changes include version bump to 0.6.1, architectural refactoring from inheritance to interface implementation, and content transformation logic with comprehensive test coverage.

Changes

Cohort / File(s)	Summary
Version & Metadata CHANGELOG.md, package.json	Version bumped to 0.6.1; changelog entry documents fix for openai_responses adapter to use Responses API native content types instead of Chat Completions types.
Core Adapter Implementation src/model.ts	OpenAIResponsesAdapter changed from extending OpenAILLMAdapter to implementing ILLMAdapter<ProviderMessage[]>. Added provider() method. Rewrote toLLMSyntax to transform content types: text → input_text, image URLs → input_image, base64 media → input_file/input_image via new private formatBase64Content() helper. System messages filtered out; unsupported content throws freeplayError.
Test Coverage test/openaiResponses.test.ts	Added comprehensive test suite validating content block type conversions: text to input_text, media URLs to input_image, base64 images/files to respective Responses API types, audio error handling, and content passthrough behavior.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• Add OpenAI Responses API adapter and developer role support #15: Modifies OpenAIResponsesAdapter's message-to-Responses formatting and content type handling in toLLMSyntax.

Suggested reviewers

• nicot

Poem

🐰 Content types shifting, from Chat to Responses so fine,
Input blocks dancing where old ones did align,
Base64 transforms with a formatter's care,
Tests validate the journey, error paths laid bare! 🎯

🚥 Pre-merge checks | ✅ 2

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically summarizes the main change: fixing the adapter to use Responses API native content types instead of Chat Completions types.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch aditya/fix-responses-api-content-types

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

test/openaiResponses.test.ts (1)

156-184: Consider adding a test for non-image URL rejection.

The implementation throws an error when slot_type is not "image" for media URLs. A test for this error path would improve coverage.

📝 Example test case

test("throws for non-image URL content", () => {
  const messages: ProviderMessage[] = [
    {
      role: "user",
      content: [
        {
          content_part_type: "media_url",
          url: "http://example.com/video.mp4",
          slot_name: "video-1",
          slot_type: "video",
        },
      ],
    },
  ];

  expect(() => adapter.toLLMSyntax(messages)).toThrow(
    "Message contains a non-image URL, but OpenAI Responses API only supports image URLs.",
  );
});

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@test/openaiResponses.test.ts` around lines 156 - 184, Add a unit test
covering the error path where a media_url has a non-"image" slot_type: call
adapter.toLLMSyntax with a message whose content includes a media_url with
slot_type "video" and assert it throws the expected message "Message contains a
non-image URL, but OpenAI Responses API only supports image URLs."; place the
test alongside existing cases in openaiResponses.test.ts and reference
adapter.toLLMSyntax to ensure the non-image URL rejection logic is exercised.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/model.ts`:
- Around line 388-407: In formatBase64Content (MediaContentBase64 handling) add
defensive validation for item.content_type before using
item.content_type.split("/")[1]; if content_type is missing or does not contain
"/", fall back to a safe mime like "application/octet-stream" and a default file
extension (e.g., "bin") when building filename and the data URL, and ensure
filename uses a sanitized slot_name; update the branch that returns input_file
to compute ext = (content_type.includes("/") ? content_type.split("/")[1] :
"bin") and use a safe contentType variable in the data URL so undefined is never
embedded.

---

Nitpick comments:
In `@test/openaiResponses.test.ts`:
- Around line 156-184: Add a unit test covering the error path where a media_url
has a non-"image" slot_type: call adapter.toLLMSyntax with a message whose
content includes a media_url with slot_type "video" and assert it throws the
expected message "Message contains a non-image URL, but OpenAI Responses API
only supports image URLs."; place the test alongside existing cases in
openaiResponses.test.ts and reference adapter.toLLMSyntax to ensure the
non-image URL rejection logic is exercised.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: c8a47841-43c7-47bb-9a09-83e006fbba42

📥 Commits

Reviewing files that changed from the base of the PR and between 737e1d5 and fd1a2c7.

⛔ Files ignored due to path filters (1)

• package-lock.json is excluded by !**/package-lock.json

📒 Files selected for processing (4)

• CHANGELOG.md
• package.json
• src/model.ts
• test/openaiResponses.test.ts