Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] parsing error when tool_calls has array in arguments #3829

Open
ianchi opened this issue Oct 23, 2024 · 0 comments
Open

[BUG] parsing error when tool_calls has array in arguments #3829

ianchi opened this issue Oct 23, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@ianchi
Copy link

ianchi commented Oct 23, 2024

Describe the bug
If the chat history rendered in the template includes an "assistant" message that has tool_calls, if any call has arguments with an array, the parsing is done incorrectly, and is impossible to include this in the chat_history

This is the error I get when the call to Azure OpenAI is made:

'Error code: 400 - {\'error\': {\'message\': "Invalid parameter: messages with role \'tool\' must be a response to a preceeding message with \'tool_calls\'.", \'type\': \'invalid_request_error\', \'param\': \'messages.[3].role\', \'code\': None}}'

But the problem is not that the message is not in response to a previous one, but that the previous message is wrongly parsed and thus not correctly seen by ChatGPT.

this is the result of prompty.parse for the offending message:

{'role': 'user', 'content': 'lista las \u00faltimas diez ...'}

{'role': 'assistant', 'content': '## tool_calls:\\n[\\n{\\'id\\': \\'call_DfeqCkPa2on8Fu2YEnwR4aWv\\', \\'function\\': {\\'name\\': \\'get_documents\\', \\'arguments\\': \\'{\\\\n  \"top_k\": 10,\\\\n  \"sort_order\": \"desc\",\\\\n  \"filter_by_metadata\": \"{\\\\\\\\\"$or\\\\\\\\\": [{\\\\\\\\\"topics\\\\\\\\\": \\\\\\\\\"precios\\\\\\\\\"}, {\\\\\\\\\"topics\\\\\\\\\": \\\\\\\\\"defectos\\\\\\\\\"}]}\"\\\\n}\\'}, \\'type\\': \\'function\\'}\\n]'}, 

{'role': 'tool', 'tool_call_id': 'call_DfeqCkPa2on8Fu2YEnwR4aWv', 'content': 'Se detallan a continuaci\u00f3n las 10 .....'}]

As you can see the ## tool_calls: was not correctly parsed as the regex used failed with the ] in the text. It was sent as part of content instead as tool_calls array.

How To Reproduce the bug
Steps to reproduce the behavior, how frequent can you experience the bug:

  1. define a tool that accepts an array and generate a prompt that makes the assistant decide to call it

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Running Information(please complete the following information):

  • Promptflow Package Version using pf -v: [e.g. 0.0.102309906]
  • Operating System: [e.g. Ubuntu 20.04, Windows 11]
  • Python Version using python --version: [e.g. python==3.10.12]

Additional context
Add any other context about the problem here.

@ianchi ianchi added the bug Something isn't working label Oct 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant