-
Notifications
You must be signed in to change notification settings - Fork 128
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Anything after <file_sep> to be stripped from completion #268
Comments
Hello, I just released a new version which adds |
@rjmacarthy What your change does is removing So now, with this change I can't use my hacky method of splitting on |
Ok I did a little research and what is needed is ability to set stop sequence on here is what works with
|
Hello @robertpiosik the change to remove the stop word is the correct approach as we don't want in the final output, in order for you to continue working with starcoder2 you need to create the correct Modelfile for Ollama and specify your stop words in the configuration. All the best. https://github.com/ollama/ollama/blob/main/docs/modelfile.md#valid-parameters-and-values |
Ok, thank you! |
Hey @robertpiosik did that work for you? Many thanks. |
I decided to use |
Ok no worries. Thanks for the help, all the best! |
Describe the bug
When using starcoder2:3b my completions contain fim tokens.
My workaround is to split output on <file_sep>.
To Reproduce
starcoder2:3b with ollama
Screenshots
The text was updated successfully, but these errors were encountered: