Skip to content

Let users supply additional CLI arguments to llama-server#103

Open
mvanhorn wants to merge 1 commit into
ggml-org:mainfrom
mvanhorn:feat/71-additional-cli-arguments
Open

Let users supply additional CLI arguments to llama-server#103
mvanhorn wants to merge 1 commit into
ggml-org:mainfrom
mvanhorn:feat/71-additional-cli-arguments

Conversation

@mvanhorn

Copy link
Copy Markdown

Summary

Adds an "Additional arguments" setting that lets you pass extra llama serve flags the GUI doesn't expose. The value is a single string the app tokenizes into argv and appends to the server launch spec after the built-in flags.

Why this matters

As raised in #71, the GUI only surfaces a handful of server options (port, idle timeout, model directory), but llama serve accepts many more (sampling, threading, and experimental llama.cpp flags). The environment-variable route doesn't help here: a macOS GUI app launched from Finder doesn't inherit the shell environment, so LLAMA_ARG_* never reaches the server. A persisted setting is the only way to get these flags through, which is the direction the issue discussion landed on.

How it works

UserSettings.customServerArguments (Llama/Settings/UserSettings.swift) is a persisted String? that follows the existing serverPort setter exactly: it trims the value, treats empty or whitespace-only as unset, skips a redundant write, and posts .LBUserSettingsDidChange so the LlamaServer observer reloads the server once when the value changes. LlamaServer.buildLaunchSpec() appends the tokenized result last, so the flags also appear in the read-only "Server command" preview for free and an unset value leaves the spec identical to before.

Tokenizing happens with a small scanner rather than a naive split so that simple single/double quoting works (a quoted path with spaces stays one argv element) and runs of whitespace never produce empty tokens. --port and --host are dropped from the user tokens on purpose: the rest of the app derives its health-check target and the menu/WebUI URLs from LlamaServer.port and resolvedHost, so a custom --port would bind the server somewhere the app isn't polling and leave it stuck loading. Every other flag passes through untouched.

The Settings row sits next to "Server port" and reuses the existing ServerPortSheet / RestoreDefaultButton pattern: a button opens an edit sheet with Save/Cancel, and a restore-default button appears only once a custom value is set.

Testing

Verified that --threads 8 --temp 0.7 appears as the trailing tokens of buildLaunchSpec().arguments and in the rendered command with a single reload on save; that a quoted path with spaces stays one element; that clearing the field reproduces the no-custom-args launch spec exactly; and that leading, trailing, and repeated internal whitespace collapse without empty tokens.

Fixes #71

@julien-c

Copy link
Copy Markdown
Contributor

Do you have a screenshot of the what the UI looks like?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow Users to Supply Additional CLI arguments to Llama Server

2 participants