feat: surface tool_calls on ChatResponse#19
Merged
Conversation
Adds a `tool_calls: Option<Value>` field to `ChatResponse` carrying
model-requested function calls in OpenAI shape — an array of
`{ id, type: "function", function: { name, arguments } }` where
`arguments` is the raw JSON string the model produced.
- OpenAI-compatible providers pass `tool_calls` through unchanged from the
assistant message. Empty / missing arrays normalize to `None` so callers
can use `if let Some(_)` reliably.
- Anthropic `tool_use` content blocks are mapped to the OpenAI shape
(the block `id`/`name`/`input` become the call id/function name/arguments).
- Gemini `functionCall` parts on the first candidate are mapped the same
way, synthesizing `call_<idx>` ids since Gemini doesn't supply them.
Tests cover all three providers plus the empty-array and missing-field
cases.
Owner
|
Sorry for the slow turnaround, this one slipped past us. Took a proper look now: clean, useful change, and it builds, tests, and clippy-passes locally. Two small things before CI goes green:
Once those land I'll approve the workflow run and we can merge. Thanks for the contribution. Gemini test — add to
|
Owner
|
Merged, thanks @ocervell — this is a clean, genuinely useful addition and the cross-provider normalization is exactly the right shape. I applied the two small follow-ups from the review directly on |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tool_calls: Option<Value>field toChatResponsecarrying model-requested function calls in OpenAI shape — an array of{ id, type: \"function\", function: { name, arguments } }whereargumentsis the raw JSON string the model produced.tool_callsthrough unchanged from the assistant message. Empty / missing arrays normalize toNoneso callers can useif let Some(_)reliably.tool_usecontent blocks are mapped to the OpenAI shape (the blockid/name/inputbecome the call id / function name / arguments).functionCallparts on the first candidate are mapped the same way, synthesizingcall_<idx>ids since Gemini doesn't supply them.Why
Building an agent loop on top of litellm-rust currently isn't possible — you can SEND
req.tools = Some(...)but the model'stool_callsreply is dropped by the OpenAI parser (OpenAIMessageonly deserializescontent+reasoning_*), so the agent never sees what the model wants to invoke.This patch closes that gap with a single new field. The shape is OpenAI's (the existing target for
ChatRequest.tools) so consumers get one dispatch path; Anthropic and Gemini are normalized through the same shape so swapping providers doesn't change call-site code.Test plan
cargo test— 69 passing (was 65, +4 new):chat_completion_surfaces_tool_calls— OpenAI tool_calls round-tripchat_completion_omits_tool_calls_when_absent— missing field staysNonechat_completion_treats_empty_tool_calls_as_none— empty array staysNonechat_completion_maps_anthropic_tool_use_to_openai_tool_calls— Anthropic mappingChatResponsegains a field with#[serde(default, skip_serializing_if = \"Option::is_none\")]; existing serialized payloads still deserialize, payloads with no tool calls serialize identically.CHANGELOG entry added under `[Unreleased]`.