Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

undefined with deepseek-lite-ggml and llama.cpp #266

Open
micsthepick opened this issue Jul 4, 2024 · 1 comment
Open

undefined with deepseek-lite-ggml and llama.cpp #266

micsthepick opened this issue Jul 4, 2024 · 1 comment

Comments

@micsthepick
Copy link

Describe the bug
any call to the chat completion api in llama.cpp returns a string of "undefined" with the DeepSeek-Coder-V2-Lite-Instruct-Q6_K.gguf model

To Reproduce
Download the model, run with the following command:
./llama-server -ngl 99 --ctx-size 16384 -m models/DeepSeek-Coder-V2-Lite-Instruct/DeepSeek-Coder-V2-Lite-Instruct-Q6_K.gguf
, add a chat provider for llamacpp at localhost:8080/v1/chat/completions.

Expected behavior
undefined is NOT repeated and the generated text makes at least some level of sense

Screenshots
image

Logging
logs.txt

API Provider
as above, llama.cpp default port on localhost:8080/v1/chat/completions

Chat or Auto Complete?
chat

Model Name
DeepSeek-Coder-V2-Lite-Instruct-Q6_K.gguf

Desktop (please complete the following information):

  • OS: Windows 11 Edu
  • Browser N/A

Additional context

@Raizo62
Copy link

Raizo62 commented Jul 8, 2024

Hi

And if you set as Provider the value "ollama" (instead of "llamacpp", even if you use llama.cpp) ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants