-
-
Notifications
You must be signed in to change notification settings - Fork 62
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unable to support the qwen model well #63
Comments
This may be due to the model I downloaded from HF using an old version of llama.cpp conversion, which seems to have been fixed in the new version |
Can you tell me if reconverting solved your problem? |
Hello, I have re exported the gguf model using llama.cpp and the issue has not been resolved. Here is the prompt template I tried: <|im_start|>system The following is the interaction mode of llama.cpp, using: == Running in interactive mode. ==
<|endoftext|><|im_start|>system
in llama.cpp The converted model works very well |
try this template
|
same problem Not working It is easy to reproduce the problem when using irregular short prompt for questioning |
I think it's all in the |
It looks like the <|im_start|> token does not appear with this prompt format. Try this. The first and last lines are empty.
|
I am using the app in testlight for testing, but there is still an issue. I am currently unable to compile this project, and I apologize for not being able to assist |
also try add <|im_start|> to reverse prompt |
I tried qwen 0.5b, 1.8b, 4b, and 7b, and only 0.5b worked properly. However, the result was not as good as the official demo. The output included the label "<| im_start |>" in the prompt, which I believe will affect the final inference result. I hope to improve it to better support the qwen series model. Thank you
The text was updated successfully, but these errors were encountered: