MCP tools not provided in prompt

[ebudan]

Thank you for this effort! It looks promising. However... I'm trying to get a handle of mcp calls on a local setup, and hitting a wall: dolphin-mcp does not build the kind of prompt I'd expect from the MCP specification.

Dolphin-mcp version is git HEAD from today, 2025-03-20, the main dependencies python=3.12.9 and openai=1.67.0.

As a test case, I've tried running a file system provider (link) against qwen2.5-coder-32b-instruct (both lmstudio/openai and ollama, here using the lmstudio output as example). Digging into the dolphin-mcp code and print-debugging a bit, I see that generate_with_openai() receives the mcp server tool list fine. However, the prompt I see coming in (lms log stream) is:

timestamp: 3/20/2025, 4:52:10 PM
type: llm.prediction.input
modelIdentifier: qwen2.5-coder-32b-instruct
modelPath: Qwen/Qwen2.5-Coder-32B-Instruct-GGUF/qwen2.5-coder-32b-instruct-q4_k_m.gguf
input: "<|im_start|>system
You are a helpful assistant with access to MCP tools. You can Automatically detect when requests require MCP server capabilities and route them to the appropriate server.<|im_end|>
<|im_start|>user
Tell me what tools you have available.<|im_end|>
<|im_start|>assistant
"

The system prompt is the naive systemMessage I specified for the model. Although the openai docs does state that functions is deprecated in favour of tools, I would have expected to see the tool description injected into the full generated prompt. The model itself should be tool capable, its default system prompt template has a corresponding section, but it does not return the tool call interim response. (Tested with multiple approaches.)

Any suggestions going forward?

[ehartford]

dolphin-mcp is a work in progress and very much v0.1.0

if you see ways it can be improved I would love to review PR's that make it better.
I will have a chance to look at it myself this weekend

[enzopoeta]

Hello everyone! My Fork has the tool call working for ollama models. Feel free to test it. I have also submitted a PR to incorporate the changes in the original project. I hope it helps! Em qui., 20 de mar. de 2025 às 13:24, ebudan ***@***.***> escreveu:

…

Thank you for this effort! It looks promising. However... I'm trying to get a handle of mcp calls on a local setup, and hitting a wall: dolphin-mcp does not build the kind of prompt I'd expect from the MCP specification. Dolphin-mcp version is git HEAD from today, 2025-03-20, the main dependencies python=3.12.9 and openai=1.67.0. As a test case, I've tried running a file system provider (link <https://github.com/mark3labs/mcp-filesystem-server>) against qwen2.5-coder-32b-instruct (both lmstudio/openai and ollama, here using the lmstudio output as example). Digging into the dolphin-mcp code and print-debugging a bit, I see that generate_with_openai() receives the mcp server tool list fine. However, the prompt I see coming in (lms log stream) is: timestamp: 3/20/2025, 4:52:10 PM type: llm.prediction.input modelIdentifier: qwen2.5-coder-32b-instruct modelPath: Qwen/Qwen2.5-Coder-32B-Instruct-GGUF/qwen2.5-coder-32b-instruct-q4_k_m.gguf input: "<|im_start|>system You are a helpful assistant with access to MCP tools. You can Automatically detect when requests require MCP server capabilities and route them to the appropriate server.<|im_end|> <|im_start|>user Tell me what tools you have available.<|im_end|> <|im_start|>assistant " The system prompt is the naive systemMessage I specified for the model. Although the openai docs <https://platform.openai.com/docs/guides/function-calling?api-mode=chat> does state that functions is deprecated in favour of tools, I would have expected to see the tool description injected into the full generated prompt. Any suggestions going forward? — Reply to this email directly, view it on GitHub <#13>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJ5AXOEHHSHUPOVH3EJXHGD2VLTTPAVCNFSM6AAAAABZN4YINGVHI2DSMVQWIX3LMV43ASLTON2WKOZSHEZTMMBTG44DKMY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***> [image: ebudan]*ebudan* created an issue (QuixiAI/dolphin-mcp#13) <#13> Thank you for this effort! It looks promising. However... I'm trying to get a handle of mcp calls on a local setup, and hitting a wall: dolphin-mcp does not build the kind of prompt I'd expect from the MCP specification. Dolphin-mcp version is git HEAD from today, 2025-03-20, the main dependencies python=3.12.9 and openai=1.67.0. As a test case, I've tried running a file system provider (link <https://github.com/mark3labs/mcp-filesystem-server>) against qwen2.5-coder-32b-instruct (both lmstudio/openai and ollama, here using the lmstudio output as example). Digging into the dolphin-mcp code and print-debugging a bit, I see that generate_with_openai() receives the mcp server tool list fine. However, the prompt I see coming in (lms log stream) is: timestamp: 3/20/2025, 4:52:10 PM type: llm.prediction.input modelIdentifier: qwen2.5-coder-32b-instruct modelPath: Qwen/Qwen2.5-Coder-32B-Instruct-GGUF/qwen2.5-coder-32b-instruct-q4_k_m.gguf input: "<|im_start|>system You are a helpful assistant with access to MCP tools. You can Automatically detect when requests require MCP server capabilities and route them to the appropriate server.<|im_end|> <|im_start|>user Tell me what tools you have available.<|im_end|> <|im_start|>assistant " The system prompt is the naive systemMessage I specified for the model. Although the openai docs <https://platform.openai.com/docs/guides/function-calling?api-mode=chat> does state that functions is deprecated in favour of tools, I would have expected to see the tool description injected into the full generated prompt. Any suggestions going forward? — Reply to this email directly, view it on GitHub <#13>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJ5AXOEHHSHUPOVH3EJXHGD2VLTTPAVCNFSM6AAAAABZN4YINGVHI2DSMVQWIX3LMV43ASLTON2WKOZSHEZTMMBTG44DKMY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[ebudan]

Confirming that enzopoeta's fork works against the MCP server I'm toying with using ollama and qwen2.5. If I can cohere enough to compare the changes and apply to the openai provider side, I'll try to make a corresponding PR.

[ebudan]

Pulling commit 5b11eb5 (also, switching model to qwq-32b and provider to lmstudio), the tool prompt is now injected.
The function parameter descriptions are however not. Looks like the lmstudio json_api.py parses them from the function type hints, so creating a tool_function(*args, **kwargs) will emit an empty parameter list in the injected JSON.

If I find a neat solution, will add as a pull request.

[ehartford]

I appreciate your contribution!