Skip to content

[Feature][Tools]: Complete Redesign of Tool Calling #22918

@robertgshaw2-redhat

Description

@robertgshaw2-redhat

🚀 The feature, motivation and pitch

  • We currently have a patchwork of support for tools in vLLM
  • We currently have regexes in tools which can cause noisy-neighbor issues in vLLM
  • We would welcome a contributor to redesign the system, improve our CI coverage, and work together with the ecosystem to ensure vLLM's support for tool calling is elite

Alternatives

No response

Additional context

No response

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions