Skip to content

feat: add remote GPU cluster support for multi-node model fitting #279

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
michaeljabbour wants to merge 5 commits into
base: main
Choose a base branch
from
Open

feat: add remote GPU cluster support for multi-node model fitting #279

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
7f4a708
feat: add remote GPU cluster support for multi-node model fitting
michaeljabbour May 3, 2026
5ff1a38
docs: note cluster-mode semantics of Hardware Simulation; clarify VRA…
michaeljabbour Aug 23, 2026
ebdaf52
fix(tui): replace CPR-dependent Terminal::clear() with same-size full…
michaeljabbour Aug 24, 2026
492784b
fix(cli): honor --ray-url on cluster init, apply hardware overrides t…
michaeljabbour Aug 24, 2026
0f20c80
fix(cli): refuse cluster specs with GPUs but unset per-GPU VRAM
michaeljabbour Aug 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions docs/tui.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,8 @@ Press `S` to open the hardware simulation popup. Override RAM, VRAM, and CPU cor

When simulation is active, a `SIM` badge appears in the system bar and status bar. The entire model table reflects the simulated hardware until you reset.

**Cluster mode (`--cluster`):** Simulation operates on the cluster's aggregate specs. The VRAM field is per-GPU and applies uniformly to every GPU across all nodes; RAM and CPU cores are cluster-wide totals. Fit and tokens/second estimates do not model interconnect overhead between nodes, so simulated multi-node throughput is optimistic. Confirming the popup without edits applies the simulation availability model (90% of total RAM) rather than the cluster profile's default 85% reservation.

### Advanced Configuration (`A`)

Press `A` to open the Advanced Configuration popup. This panel lets you tune the parameters behind TPS estimation, run mode penalties, and composite scoring — addressing [issue #449](https://github.com/AlexsJones/llmfit/issues/449) where tok/s was overestimated for certain models (e.g., Qwen3 30B).
Expand Down
1 change: 1 addition & 0 deletions llmfit-core/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
yaml_serde = "0.10.4"
sysinfo = "0.39"
toml = "0.8"
ureq = { version = "3.2", features = ["json"] }
which = "8.0.2"

Expand Down
Loading