feat(llm): virtualize provider interface behind a normalized Adapter - #1683
feat(llm): virtualize provider interface behind a normalized Adapter#1683dsapandora wants to merge 18 commits into
Conversation
📝 WalkthroughWalkthroughThe PR adds normalized streaming adapters and content parsing, routes ChatBase execution through them, enables opt-in Anthropic extended thinking for reasoning models, exposes the setting in service profiles, and synchronizes the field based on model capabilities. ChangesUnified LLM streaming and Anthropic thinking
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Caller
participant ChatBase
participant Adapter
participant Provider
Caller->>ChatBase: chat_string(prompt)
ChatBase->>Adapter: stream(prompt)
Adapter->>Provider: consume provider stream
Provider-->>Adapter: thinking and text events
Adapter-->>ChatBase: normalized events
ChatBase-->>Caller: callbacks and final text
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 Biome (2.5.5)nodes/src/nodes/llm_anthropic/services.jsonFile contains syntax errors that prevent linting: Line 2: Expected a property but instead found '//'.; Line 6: End of file expected; Line 6: End of file expected; Line 6: End of file expected; Line 6: End of file expected; Line 11: End of file expected; Line 11: End of file expected; Line 11: End of file expected; Line 11: End of file expected; Line 16: End of file expected; Line 16: End of file expected; Line 16: End of file expected; Line 16: End of file expected; Line 22: End of file expected; Line 22: End of file expected; Line 22: End of file expected; Line 22: End of file expected; Line 28: End of file expected; Line 28: End of file expected; Line 28: End of file expected; Line 28: End of file expected; Line 34: End of file expected; Line 34: End of file expected; Line 34: End of file expected; Line 34: End of file expected; Line 40: End of file expected; Line 40: End of file expected; Line 40: End of file expected; Line 40: End of file expected; Line 45: End of file expected; Li ... [truncated 5588 characters] ... erty but instead found '// openrouter'.; Line 279: expected Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🤖 Internal: Discord sync markerAuto-managed by the Discord notification workflow. Stores the linked Discord message ID and forum thread ID. Do not edit or delete. |
Pull the provider-shape → (text, reasoning) walker out of chat_string into _make_stream_content_parser, pinned by a characterization test. No behavior change.
Event (thinking/text/done) + Adapter Protocol with opaque history items — the seam ChatBase will consume. Not wired yet. RFC: discussion #1679.
Relocate _make_stream_content_parser / _make_think_tag_splitter into llm_adapter so adapters can reuse them without a chat↔adapter import cycle.
Wraps a LangChain chat model as an Event stream (reusing the shared parser); covers the non-reasoning providers. done.items is the assistant text turn.
Streams the Messages API (thinking/text deltas → Events); done.items is the assembled content with signatures intact — appended to history verbatim.
Streams the Responses API (reasoning/text deltas → Events); done.items are the output items with encrypted reasoning, extended into history verbatim.
Consumes an adapter's Event stream into the existing on_chunk/on_reasoning_chunk callbacks and returns (answer_text, opaque done.items). Not wired yet.
chat_string now streams via LangChainAdapter + drive_adapter instead of the inline walker; stop sequences and finish_reason preserved. Native/Responses paths unchanged.
_chat returned Anthropic's typed-block list verbatim, crashing agents/expectJson on .strip(); flatten_content keeps text blocks, drops thinking. Fixes the #1658 class.
NativeAnthropicAdapter yields Events over the same payload + create(stream=True) path; try_anthropic_native_chat_stream now drives it. Behavior preserved.
NativeOpenAIResponsesAdapter yields Events over responses.create(stream=True); _chat_string_responses drives it, keeping the non-streaming fallback. Behavior preserved.
_chat now drains the adapter (same iterator/normalization as streaming) instead of invoke+flatten, so content handling is one mechanism. Fallback kept for stream-less backends.
2f016e1 to
46d6a16
Compare
…gate Replace the interim hard-off (#1681) with an opt-in `extendedThinking` node toggle (default off). Thinking is no longer baked into the client; the native adapter adds it per call, so it rides the interactive streaming path only — agent/expectJson never gets it.
…ading _stream_anthropic_messages_api folded into NativeAnthropicAdapter on this branch; update the stop-threading test to drive the adapter. Same payload wiring, no behavior change.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@nodes/src/nodes/llm_anthropic/services.json`:
- Around line 508-515: Remove "extendedThinking" from the properties array for
the anthropic.claude-3-haiku entry, leaving its API key and modelSource
properties unchanged.
In `@packages/ai/src/ai/common/chat.py`:
- Around line 406-411: Update the native OpenAI Responses path around
NativeOpenAIResponsesAdapter and drive_adapter to raise an exception when no
text is produced, matching the Anthropic native stream behavior. Ensure this
occurs before on_finish and return, so failed streams with finish_reason='error'
enter the existing fallback logic, while successful responses containing text
retain the current flow.
In `@packages/ai/src/ai/common/llm_adapter.py`:
- Around line 173-278: Remove the unused AnthropicAdapter and OpenAIAdapter
classes from this module, since production routing uses NativeAnthropicAdapter
and NativeOpenAIResponsesAdapter instead. Preserve LangChainAdapter and any
shared functionality that is actively referenced.
- Around line 288-322: Update NativeOpenAIResponsesAdapter.stream() at the
responses.create(...) call to pass store=False explicitly, matching
OpenAIAdapter’s stateless behavior. Add or synchronize the nearby
rationale/documentation so it states that Responses API requests and responses
should not be stored.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 1739171e-5c99-4817-aea9-b8ba208604b8
📒 Files selected for processing (15)
nodes/src/nodes/llm_anthropic/anthropic.pynodes/src/nodes/llm_anthropic/services.jsonnodes/test/agent_crewai/test_stop_words.pypackages/ai/src/ai/common/chat.pypackages/ai/src/ai/common/llm_adapter.pypackages/ai/src/ai/common/llm_native_stream.pypackages/ai/tests/ai/common/test_chat_content.pypackages/ai/tests/ai/common/test_chat_string_wiring.pypackages/ai/tests/ai/common/test_llm_adapter.pypackages/ai/tests/ai/common/test_llm_adapter_anthropic.pypackages/ai/tests/ai/common/test_llm_adapter_drive.pypackages/ai/tests/ai/common/test_llm_adapter_langchain.pypackages/ai/tests/ai/common/test_llm_adapter_openai.pypackages/ai/tests/ai/common/test_native_anthropic_adapter.pypackages/ai/tests/ai/common/test_native_openai_responses_adapter.py
The model sync now adds the extendedThinking UI toggle to reasoning-capable profiles (incl. models added on future syncs) and removes it from non-reasoning ones — the UI stays self-maintaining. Also drops the inert toggle from claude-3-haiku (non-reasoning).
…nses fallback - Remove unused AnthropicAdapter/OpenAIAdapter (.stream() wrappers) + their tests; production routes through the Native* adapters. - NativeOpenAIResponsesAdapter now passes store=False (no server-side retention). - _chat_string_responses raises on empty output so it falls back (matching the Anthropic path) instead of returning '' silently.
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
nodes/src/nodes/llm_anthropic/services.json (1)
319-366: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick winPlace
extendedThinkingbeforellm.cloud.modelSource.The affected arrays currently append
extendedThinkingafterllm.cloud.modelSource, buttools/sync_models/src/core/patcher.pyLines 338-345 andtools/sync_models/test/test_extended_thinking_field.pyLines 26-28 requirellm.cloud.modelSourceto be last. Reorder every affected array so the checked-in schema matches the synchronization contract.Proposed fix
- "llm.cloud.modelSource", - "extendedThinking" + "extendedThinking", + "llm.cloud.modelSource"Also applies to: 500-505, 519-608
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@nodes/src/nodes/llm_anthropic/services.json` around lines 319 - 366, Reorder the properties arrays for every affected Anthropic model entry, including anthropic.custom and the listed Claude models, so extendedThinking appears before llm.cloud.modelSource and llm.cloud.modelSource remains the final property. Apply this consistently across all occurrences covered by the review.tools/sync_models/src/core/patcher.py (1)
280-305: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDocument the new
profilesargument.
_repair_field_objectsnow acceptsprofilesand uses it to add or removeextendedThinking, but the docstring only documentsfields. Add the argument and its capability-aware behavior.Proposed documentation update
Args: fields: The ``"fields"`` dict to mutate in-place + profiles: Optional complete profile mapping used to determine whether + each model supports reasoning🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tools/sync_models/src/core/patcher.py` around lines 280 - 305, Update the _repair_field_objects docstring to document the profiles argument, including that it is used to determine capability-aware additions or removals of the extendedThinking field. Also describe the expected optional mapping shape or absence behavior consistently with the function signature, without changing implementation logic.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@nodes/src/nodes/llm_anthropic/services.json`:
- Around line 319-366: Reorder the properties arrays for every affected
Anthropic model entry, including anthropic.custom and the listed Claude models,
so extendedThinking appears before llm.cloud.modelSource and
llm.cloud.modelSource remains the final property. Apply this consistently across
all occurrences covered by the review.
In `@tools/sync_models/src/core/patcher.py`:
- Around line 280-305: Update the _repair_field_objects docstring to document
the profiles argument, including that it is used to determine capability-aware
additions or removals of the extendedThinking field. Also describe the expected
optional mapping shape or absence behavior consistently with the function
signature, without changing implementation logic.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: dc76eb4a-7050-44b9-b7c4-842bcc34a8f3
📒 Files selected for processing (6)
nodes/src/nodes/llm_anthropic/services.jsonpackages/ai/src/ai/common/chat.pypackages/ai/src/ai/common/llm_adapter.pytools/sync_models/src/core/patcher.pytools/sync_models/src/providers/base.pytools/sync_models/test/test_extended_thinking_field.py
…ine the field The capabilities-aware repair only adds/manages extendedThinking on nodes whose fields define it (today: llm_anthropic), so it never leaks into other providers (openai, gemini, …) that neither define nor read the flag.
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tools/sync_models/src/core/patcher.py (1)
283-305: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDocument the new toggle behavior in both helper docstrings.
_repair_field_objectsnow acceptsprofiles, but itsArgssection omits that parameter._update_fields_for_addedalso conditionally addsextendedThinking, while its docstring still describes only the original three entries. Document the profile shape,Nonebehavior, and conditional property ordering.Also applies to: 357-390
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tools/sync_models/src/core/patcher.py` around lines 283 - 305, Update the docstrings for _repair_field_objects and _update_fields_for_added to document the profiles parameter, including its expected shape and behavior when it is None. In _update_fields_for_added, describe that extendedThinking is conditionally added and specify the resulting property ordering, while preserving documentation of the existing entries.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tools/sync_models/test/test_extended_thinking_field.py`:
- Around line 45-54: Update test_repair_skips_custom to include a custom object
whose properties omit extendedThinking, then run _repair_field_objects and
assert the property remains absent. Preserve the existing custom-object coverage
while ensuring the test fails if repair incorrectly adds the toggle to custom
objects.
---
Outside diff comments:
In `@tools/sync_models/src/core/patcher.py`:
- Around line 283-305: Update the docstrings for _repair_field_objects and
_update_fields_for_added to document the profiles parameter, including its
expected shape and behavior when it is None. In _update_fields_for_added,
describe that extendedThinking is conditionally added and specify the resulting
property ordering, while preserving documentation of the existing entries.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 2f0b6ef9-caac-40ab-92b9-09dd682dfa17
📒 Files selected for processing (2)
tools/sync_models/src/core/patcher.pytools/sync_models/test/test_extended_thinking_field.py
| def test_repair_skips_custom(): | ||
| fields = { | ||
| **_FIELD_DEF, | ||
| 'ns.custom': { | ||
| 'object': 'custom', | ||
| 'properties': ['llm.cloud.apikey', 'extendedThinking', 'llm.cloud.modelSource'], | ||
| }, | ||
| } | ||
| _repair_field_objects(fields, {'custom': {}}) | ||
| assert 'extendedThinking' in fields['ns.custom']['properties'] |
There was a problem hiding this comment.
🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win
Make the custom-object test prove the skip behavior.
Because the fixture already contains extendedThinking, this test would pass even if the repair logic incorrectly added the toggle to custom objects. Also test a custom object without the property and assert that it remains absent.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tools/sync_models/test/test_extended_thinking_field.py` around lines 45 - 54,
Update test_repair_skips_custom to include a custom object whose properties omit
extendedThinking, then run _repair_field_objects and assert the property remains
absent. Preserve the existing custom-object coverage while ensuring the test
fails if repair incorrectly adds the toggle to custom objects.
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
nodes/src/nodes/llm_anthropic/services.json (1)
319-367: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winKeep
llm.cloud.modelSourceas the final property.The synchronization contract in
tools/sync_models/src/core/patcher.py:283-354explicitly keepsllm.cloud.modelSourcelast, but these new arrays appendextendedThinkingafter it. Reorder the entries consistently to avoid schema/UI ordering being repaired only later.Proposed fix
"properties": [ "llm.cloud.apikey", - "llm.cloud.modelSource", - "extendedThinking" + "extendedThinking", + "llm.cloud.modelSource" ]Apply this ordering to every affected profile.
Also applies to: 500-506, 515-609
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@nodes/src/nodes/llm_anthropic/services.json` around lines 319 - 367, Reorder the properties arrays for every affected Anthropic profile, including anthropic.custom and the listed Claude model profiles, so extendedThinking appears before llm.cloud.modelSource and llm.cloud.modelSource remains the final entry. Preserve all existing properties and apply the same ordering consistently across the affected profiles.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@nodes/src/nodes/llm_anthropic/services.json`:
- Around line 319-367: Reorder the properties arrays for every affected
Anthropic profile, including anthropic.custom and the listed Claude model
profiles, so extendedThinking appears before llm.cloud.modelSource and
llm.cloud.modelSource remains the final entry. Preserve all existing properties
and apply the same ordering consistently across the affected profiles.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: e8289913-f116-420b-8a5c-dde9ee339616
📒 Files selected for processing (1)
nodes/src/nodes/llm_anthropic/services.json
1 participant
Summary
Virtualizes the LLM provider interface behind a normalized
Event/Adapterlayer soChatBasenever touches provider-native content shapes (RFC — discussion #1679). Single-turn.Event(thinking|text|done)+ anAdapterprotocol that owns provider-nativehistorywith opaqueitems. Adrive_adaptershim fans the Event stream into the existingon_chunk/on_reasoning_chunkcallbacks, so the node / SSEthinkingsurface is unchanged.LangChainAdapter(the ~18 non-reasoning providers, no raw-SDK rewrite),NativeAnthropicAdapter(Messages API),NativeOpenAIResponsesAdapter(Responses API). Every provider path now flows through the same interface.expectJson(agent) path returned Anthropic's typed-block list verbatim and crashed on.strip()._chatnow drains the adapter and returns a plain string; content handling is one mechanism on every path (the "unify" —_chat_with_retries's invoke branch is gone).extendedThinkingtoggle (default off). Thinking is no longer baked into the client; the native adapter adds it per call, so it rides the interactive streaming path only — agent /expectJsoncalls never get it (context gate). Model-gated to reasoning-capable models.isinstance(content, list), thinking/text/signature block parsing, betas,reasoning_content, Responses vs Messages) move into the adapters; the stream content parser is a shared, tested seam.Type
refactor + feat + fix
Testing
drive_adaptershim,chat_stringwiring, native adapters, per-call thinking injection, non-streaming drain.packages/ai/tests/ai/commonsuite green (605 passed), no regressions.Checklist
develop: thinking stays off by default (as in fix(llm_anthropic): disable extended thinking at the node (interim stop-gap) #1681); this adds the opt-in toggle.Notes
tool_use. Capture ofdone.itemsis already structural viadrive_adapter.Linked Issue
Fixes #1658
Summary by CodeRabbit