Let users supply additional CLI arguments to llama-server#103
Open
mvanhorn wants to merge 1 commit into
Open
Conversation
Contributor
|
Do you have a screenshot of the what the UI looks like? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an "Additional arguments" setting that lets you pass extra
llama serveflags the GUI doesn't expose. The value is a single string the app tokenizes into argv and appends to the server launch spec after the built-in flags.Why this matters
As raised in #71, the GUI only surfaces a handful of server options (port, idle timeout, model directory), but
llama serveaccepts many more (sampling, threading, and experimental llama.cpp flags). The environment-variable route doesn't help here: a macOS GUI app launched from Finder doesn't inherit the shell environment, soLLAMA_ARG_*never reaches the server. A persisted setting is the only way to get these flags through, which is the direction the issue discussion landed on.How it works
UserSettings.customServerArguments(Llama/Settings/UserSettings.swift) is a persistedString?that follows the existingserverPortsetter exactly: it trims the value, treats empty or whitespace-only as unset, skips a redundant write, and posts.LBUserSettingsDidChangeso theLlamaServerobserver reloads the server once when the value changes.LlamaServer.buildLaunchSpec()appends the tokenized result last, so the flags also appear in the read-only "Server command" preview for free and an unset value leaves the spec identical to before.Tokenizing happens with a small scanner rather than a naive split so that simple single/double quoting works (a quoted path with spaces stays one argv element) and runs of whitespace never produce empty tokens.
--portand--hostare dropped from the user tokens on purpose: the rest of the app derives its health-check target and the menu/WebUI URLs fromLlamaServer.portandresolvedHost, so a custom--portwould bind the server somewhere the app isn't polling and leave it stuck loading. Every other flag passes through untouched.The Settings row sits next to "Server port" and reuses the existing
ServerPortSheet/RestoreDefaultButtonpattern: a button opens an edit sheet with Save/Cancel, and a restore-default button appears only once a custom value is set.Testing
Verified that
--threads 8 --temp 0.7appears as the trailing tokens ofbuildLaunchSpec().argumentsand in the rendered command with a single reload on save; that a quoted path with spaces stays one element; that clearing the field reproduces the no-custom-args launch spec exactly; and that leading, trailing, and repeated internal whitespace collapse without empty tokens.Fixes #71