Skip to content

Conversation

nshkrdotcom
Copy link

This PR resolves two issues related to the Gemini adapter, specifically when operating in mode: :json_schema:

  1. Robust Stream Chunk Parsing (Fixes FunctionClauseError):

    • The Instructor.Adapters.Gemini.parse_stream_chunk_for_mode/2 function has been updated with a fallback clause for the :json_schema mode.
    • This new clause gracefully handles non-data-bearing stream chunks (such as metadata or "STOP" signals from the Gemini API) by returning an empty string. These empty strings are subsequently filtered out by downstream processing, preventing crashes during streaming.
    • This fixes failures previously seen in tests like test :gemini json_schema gemini-2.5-flash-preview-04-17 stream partial array of objects.
  2. Compliance with Gemini's responseSchema for Ecto :map Types:

    • Gemini's responseSchema (in json_schema mode) requires that any nested object definition must include a non-empty properties field. It does not appear to support additionalProperties for defining generic maps with arbitrary keys in this mode. This makes Ecto's generic :map and {:map, type} types incompatible if directly translated.
    • The AllEctoTypes schema within test/instructor_test.exs has been modified to use conditional compilation:
      • When a test iteration targets the Gemini adapter with mode: :json_schema, AllEctoTypes now defines its nested_object and nested_object_two fields using embeds_one. These embedded schemas (InstructorTest.HelperSchemas.NestedObject and InstructorTest.HelperSchemas.NestedObjectTwo) have explicitly defined properties, thus satisfying Gemini's API requirements.
      • For all other adapter/mode combinations, AllEctoTypes will continue to use the standard field :nested_object, :map and field :nested_object_two, {:map, :string}. This allows testing the handling of generic map types with providers that support them.
    • The expected data and the user prompt within the "all ecto types" test are also dynamically adjusted to match the schema variant being used for that specific test run.
    • The client-side error message in Instructor.Adapters.Gemini.normalize_json_schema/1 (which raises if a schema would result in an object with empty properties) has been slightly reworded for better clarity regarding the use of "naked :map type". This raise correctly prevents sending an invalid schema to Gemini.
    • This change resolves test failures such as test :gemini json_schema gemini-2.5-flash-preview-04-17 all ecto types.

This approach ensures that the Gemini adapter functions correctly for streaming and handles its specific schema requirements gracefully, while also allowing the test suite to accurately test different schema interpretations across various providers in the future.

Closes #104.

NSHkr added 2 commits May 13, 2025 13:35
- Adds fallback for non-data chunks in Gemini :json_schema streaming,
  preventing FunctionClauseErrors.
- Updates AllEctoTypes test to use conditional schema definition:
  - For Gemini :json_schema, uses embeds_one for nested objects
    to comply with API's non-empty 'properties' requirement.
  - For other adapters/modes, retains generic :map type for testing.
- Slightly rewords Gemini adapter's client-side schema validation error
  for clarity regarding naked :map types.

This ensures Gemini :json_schema tests pass for streaming and complex
types, while maintaining appropriate testing for other providers.

Addresses thmsmlr#104.
- Added `:json` to `@supported_modes` in `lib/instructor/adapters/gemini.ex`.
- Updated `chat_completion` to set `response_mime_type: "application/json"` and remove `response_format` for `:json` mode.
- Implemented `parse_response_for_mode/2` and `parse_stream_chunk_for_mode/2` for `:json` mode to handle JSON decoding and streaming.
- Included preparatory `:tools` mode handling in `chat_completion` (not fully implemented).
- Updated tests in `test/instructor_test.exs` to cover `:json` mode for Gemini.

Note: One streaming test (`streams arrays one at a time`) fails due to invalid JSON output; this will be addressed in a follow-up issue.

Fixes thmsmlr#106
@nshkrdotcom nshkrdotcom changed the title Fix: Gemini Adapter Stream Handling and :map Type Compliance for json_schema Fix: Gemini Adapter Stream Handling and :map Type Compliance for json_schema; Add JSON Mode Support for Gemini Adapter May 14, 2025
@nshkrdotcom
Copy link
Author

Description:

This PR implements :json mode support for the Gemini adapter, enabling JSON response generation without a strict schema, as requested in #<ISSUE_NUMBER>.

Changes:

  • Updated lib/instructor/adapters/gemini.ex:
    • Added :json to @supported_modes (now [:json, :json_schema]).
    • Modified chat_completion to handle :json mode by setting generationConfig with response_mime_type: "application/json" and removing response_format to prevent 400 errors.
    • Added parse_response_for_mode/2 for :json mode to decode JSON text using Jason.decode/1.
    • Added parse_stream_chunk_for_mode/2 for :json mode to handle streaming JSON chunks.
    • Included preparatory changes for :tools mode in chat_completion (not fully implemented; will be completed later).
  • Updated test/instructor_test.exs to include :json mode tests for the Gemini adapter (model: gemini-2.5-flash-preview-04-17).
  • Updated .gitignore to exclude additional development directories (.elixir_ls/, .github/, .vscode/).

Testing:

  • Ran mix test --only adapter:gemini test/instructor_test.exs.
  • All :json mode tests pass except for one streaming test (streams arrays one at a time), which fails due to invalid JSON output from the Gemini model (e.g., "1732-02-22" pulsed date). This is documented in the issue for future resolution.
  • Verified that non-streaming :json mode tests (e.g., "basic ecto model," "all ecto types") pass successfully.

Known Issues:

  • The streaming test failure will be addressed in a follow-up issue by improving prompt instructions or cleaning streamed chunks.
  • The :tools mode changes are included but not fully functional; they will be completed in a separate PR.

Related Issue:

Closes #106

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gemini Adapter: Handle Non-Data Stream Chunks and Clarify :map Type Limitations in json_schema Mode

1 participant