-
Notifications
You must be signed in to change notification settings - Fork 43.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(forge/llm): Add LlamafileProvider
#7091
base: master
Are you sure you want to change the base?
feat(forge/llm): Add LlamafileProvider
#7091
Commits on Apr 18, 2024
-
Add minimal implementation of LlamafileProvider, a new ChatModelProvi…
…der for llamafiles. Currently it just extends OpenAIProvider and only overrides methods that are necessary to get the system to work at a basic level. Update ModelProviderName schema and config/configurator so that app startup using this provider is handled correctly. Add 'mistral-7b-instruct-v0' to OpenAIModelName/OPEN_AI_CHAT_MODELS registries.
Configuration menu - View commit details
-
Copy full SHA for 03d8e1e - Browse repository at this point
Copy the full SHA 03d8e1eView commit details -
Adapt model prompt message roles to be compatible with the Mistral-7b…
…-Instruct chat template, which supports the 'user' & 'assistant' roles but does not support the 'system' role.
Configuration menu - View commit details
-
Copy full SHA for ed1dfd0 - Browse repository at this point
Copy the full SHA ed1dfd0View commit details -
In
OpenAIProvider
, change methodscount_message_tokens
, `count_to……kens`, and `get_tokenizer` from classmethods so I can override them in LlamafileProvide (and so I can access instance instance attributes from inside them). Implement class `LlamafileTokenizer` that calls the llamafile server's `/tokenize` API endpoint.
Configuration menu - View commit details
-
Copy full SHA for c56c290 - Browse repository at this point
Copy the full SHA c56c290View commit details -
Configuration menu - View commit details
-
Copy full SHA for 234d059 - Browse repository at this point
Copy the full SHA 234d059View commit details -
add README for llamafile integration including setup instruction + no…
…tes on the integration; add helper scripts for downloading/running a llamafile + example env file.
Configuration menu - View commit details
-
Copy full SHA for 05d2b81 - Browse repository at this point
Copy the full SHA 05d2b81View commit details
Commits on Apr 19, 2024
-
simplify mistral message handling; set seed=0 in chat completion kwar…
…gs for reproducibility
Configuration menu - View commit details
-
Copy full SHA for 1cd3e8b - Browse repository at this point
Copy the full SHA 1cd3e8bView commit details -
set mistral max_tokens to actual value configured in the model and ch…
…ange serve.sh to use model's full context size (this does not seem to cause OOM errors, surpisingly).
Configuration menu - View commit details
-
Copy full SHA for dc36c69 - Browse repository at this point
Copy the full SHA dc36c69View commit details -
Configuration menu - View commit details
-
Copy full SHA for e426766 - Browse repository at this point
Copy the full SHA e426766View commit details
Commits on May 25, 2024
-
Configuration menu - View commit details
-
Copy full SHA for d63aa23 - Browse repository at this point
Copy the full SHA d63aa23View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7e7037d - Browse repository at this point
Copy the full SHA 7e7037dView commit details
Commits on May 30, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 3c1f283 - Browse repository at this point
Copy the full SHA 3c1f283View commit details
Commits on May 31, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 5d0f8b0 - Browse repository at this point
Copy the full SHA 5d0f8b0View commit details -
Create
BaseOpenAIProvider
with common functionality from `OpenAIPro……vider`, `GroqProvider` and `LlamafileProvider` and rebase the latter three on `BaseOpenAIProvider`
Configuration menu - View commit details
-
Copy full SHA for 960155a - Browse repository at this point
Copy the full SHA 960155aView commit details
Commits on Jun 2, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 7aed930 - Browse repository at this point
Copy the full SHA 7aed930View commit details
Commits on Jun 3, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 02d0691 - Browse repository at this point
Copy the full SHA 02d0691View commit details -
Configuration menu - View commit details
-
Copy full SHA for f53c2de - Browse repository at this point
Copy the full SHA f53c2deView commit details -
Configuration menu - View commit details
-
Copy full SHA for f78ad94 - Browse repository at this point
Copy the full SHA f78ad94View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1a00ecf - Browse repository at this point
Copy the full SHA 1a00ecfView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3c8bf3c - Browse repository at this point
Copy the full SHA 3c8bf3cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 65433ba - Browse repository at this point
Copy the full SHA 65433baView commit details
Commits on Jun 14, 2024
-
Configuration menu - View commit details
-
Copy full SHA for bc372cb - Browse repository at this point
Copy the full SHA bc372cbView commit details -
Configuration menu - View commit details
-
Copy full SHA for e1bcb03 - Browse repository at this point
Copy the full SHA e1bcb03View commit details -
Configuration menu - View commit details
-
Copy full SHA for df3278f - Browse repository at this point
Copy the full SHA df3278fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6858b22 - Browse repository at this point
Copy the full SHA 6858b22View commit details -
Configuration menu - View commit details
-
Copy full SHA for d73a98c - Browse repository at this point
Copy the full SHA d73a98cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 4d64b45 - Browse repository at this point
Copy the full SHA 4d64b45View commit details -
Configuration menu - View commit details
-
Copy full SHA for e5c5163 - Browse repository at this point
Copy the full SHA e5c5163View commit details
Commits on Jun 20, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 3cd7b0e - Browse repository at this point
Copy the full SHA 3cd7b0eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 0e081f4 - Browse repository at this point
Copy the full SHA 0e081f4View commit details
Commits on Jun 21, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 529314e - Browse repository at this point
Copy the full SHA 529314eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 072e674 - Browse repository at this point
Copy the full SHA 072e674View commit details -
Configuration menu - View commit details
-
Copy full SHA for 01372d1 - Browse repository at this point
Copy the full SHA 01372d1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 63fe5b5 - Browse repository at this point
Copy the full SHA 63fe5b5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 271e59b - Browse repository at this point
Copy the full SHA 271e59bView commit details -
Configuration menu - View commit details
-
Copy full SHA for aecc363 - Browse repository at this point
Copy the full SHA aecc363View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9ee1e8f - Browse repository at this point
Copy the full SHA 9ee1e8fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 242753e - Browse repository at this point
Copy the full SHA 242753eView commit details
Commits on Jun 24, 2024
-
Configuration menu - View commit details
-
Copy full SHA for e8905d1 - Browse repository at this point
Copy the full SHA e8905d1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3621be0 - Browse repository at this point
Copy the full SHA 3621be0View commit details -
Configuration menu - View commit details
-
Copy full SHA for f33c2d2 - Browse repository at this point
Copy the full SHA f33c2d2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 75e0301 - Browse repository at this point
Copy the full SHA 75e0301View commit details -
Configuration menu - View commit details
-
Copy full SHA for deb7d11 - Browse repository at this point
Copy the full SHA deb7d11View commit details -
Configuration menu - View commit details
-
Copy full SHA for 74923f1 - Browse repository at this point
Copy the full SHA 74923f1View commit details