Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEAT]: Disable Temperature Setting for DeepSeek-R1 (deepseek-reasoner) Model #2995

Closed
spencerthayer opened this issue Jan 20, 2025 · 3 comments
Assignees
Labels
bug Something isn't working Integration Request Request for support of a new LLM, Embedder, or Vector database

Comments

@spencerthayer
Copy link

spencerthayer commented Jan 20, 2025

When using the DeepSeek-R1 model (identified as deepseek-reasoner in AnythingLLM), the following error occurs, preventing successful message processing:

Could not respond to message.

400 deepseek-reasoner does not support the parameter `temperature`

This error arises because the DeepSeek-R1 model, as documented in the official DeepSeek-R1 repository (https://github.com/deepseek-ai/DeepSeek-R1#1-introduction), is specifically trained using Reinforcement Learning (RL) and does not support the temperature parameter. Attempting to include this parameter in API requests to the model results in the reported 400 error.

Currently, AnythingLLM presents the temperature setting as a configurable option in the UI for all language models, including deepseek-reasoner. This feature request aims to resolve this incompatibility and improve the user experience by disabling or hiding the temperature setting when the deepseek-reasoner model is selected.

@timothycarambat
Copy link
Member

timothycarambat commented Jan 20, 2025

@spencerthayer I don't know if you generate these issues programmatically or with AI, but please don't. We review every issue and having to parse though 99% noise and suggested changes that reference files that do not even exist, are in a format that does not apply to the codebase, or otherwise really amounts to us just spending more time looking at and triaging an issue that we should.

Next time, just use the bug template and any relevant information for replication. Anyone who picks up the issue would easily be able to to repro and identify the root cause from just that information. No need to propose changes unless it comes with an associated PR.

@timothycarambat timothycarambat added bug Something isn't working Integration Request Request for support of a new LLM, Embedder, or Vector database and removed enhancement New feature or request feature request labels Jan 20, 2025
@shatfield4
Copy link
Collaborator

@spencerthayer is this still an issue for you? Are you running this locally or through their API?

I am trying to repro and I am unable to get that error with the deepseek-reasoner model via their API. I've tried modifying the default temperature as well and it responds normally for me.

@spencerthayer
Copy link
Author

No problem. I've booted up a dev fork.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Integration Request Request for support of a new LLM, Embedder, or Vector database
Projects
None yet
Development

When branches are created from issues, their pull requests are automatically linked.

3 participants