Tool calls stopped working when switching from gpt-4o to anthropic claude-sonnet

[d-ivashchuk]

Description

I couldn't find it in the docs if tool calling is properly supported for anthropic models, but I don't have any type errors or runtime errors so I assumed it should work.

I changed from gpt-4o to claude-sonnet and sudennly tool calls are being ignored all togehter.

  ```
  model: openai("gpt-4o")
  ```
  to
  ``` 
  model: anthropic("claude-3-5-sonnet-20240620")
  ```

@lgrammel thanks a lot for your wonderful work maintaining ai-sdk, I hope there is some easy workaround for this if tools are not supported in anthropic 🙇🏼‍♂️

Code example

const result = await streamText({
      model: anthropic("claude-3-5-sonnet-20240620"),
      system: generateStreamInitPrompt({ projectSettings, project, chat }),
      messages: [...req.data.messages.slice(1)],
      tools: {
        finaliseLead: tool({
          description:
            "When user shares their email thank them for that and tell that we will contact them soon",
          parameters: z.object({}),
          execute: async () => {
            try {
              console.log("Executing tool for finaliseLead");
              const { object } = await generateObject({
                model: openai("gpt-4o"),
                schema: finaliseLeadToolSchema,
                prompt: generateFinaliseLeadToolPrompt({
                  messages: req.data.messages,
                }),
              });

              return {
                role: "assistant",
                content:
                  "Thank you for sharing your contact details. We will contact you soon.",
              };
            } catch (error) {
              console.log("There was an error in finalse lead tool");
              console.log(error);
            }
          },
        }),
      },
    });

Additional context

No response

[lgrammel]

Strange! it should work, can you check this demo: https://github.com/vercel/ai/blob/main/examples/ai-core/src/generate-text/anthropic-chatbot-with-tools.ts

However, there is a bug for anthropic around empty parameters: #2103 (potential duplicate)

[d-ivashchuk]

Don't think it's the same as in #2103 as I don't have any errors in console, I get response from the basic streamText instead of going a tool route. I saw an issue about debugging tool calls #2102, maybe an improvement in this area would help to at least see what is going on.

I will try to experiment with it a bit and get back to you if I find out anything new. Thanks for fast response and confirming that it should work!

[lgrammel]

@d-ivashchuk oh - that means that the anthropic model did not call the tool as expected. This is typically a prompt engineering issue. Maybe changing "description" to some definition of when the model should call the tool might help?

[d-ivashchuk]

I added optional param and tool started to be executed. You might have been right with the first bug reference in your message.

Funnily enough before with gpt-4o when tool had been executed there was no generation running. Now I have both an answer from the text stream and also a tool execution

[anikiyevichm]

Funnily enough before with gpt-4o when tool had been executed there was no generation running. Now I have both an answer from the text stream and also a tool execution

i have the same picture, but it's look like a bug for me. the done field does not appear in text generation, but the expected tool is called instead. So i see unfinished text generation process and tool calling after.

expected behavior is that text generation will not be called at all

[d-ivashchuk]

yeah, before I switched from gpt-4o it was exactly the behaviour that you described @anikiyevichm(the correct one with no text generation called at all)

Again it would be super handy to see what's going on behind the scenes in tool calling and get some debug logs to make it more prominent!