Forward guest GUI apps to the host: --x11 and --waypipe, fully automated - #665
Open
pumphaus wants to merge 4 commits into
Open
Forward guest GUI apps to the host: --x11 and --waypipe, fully automated#665pumphaus wants to merge 4 commits into
pumphaus wants to merge 4 commits into
Conversation
Contributor
|
Fascinating approach. Would like to test this, please address merge conflicts. With this successful + vulkan support, by all means we have a accessible way to run steam games on macOS. |
Author
Only sort of. Both the X11 support and Waypipe only push bytes through a vsock. There's not direct rendering from the GPU to a screen which you likely need to get reasonable performance. There's no shared memory or fd-passing between VM and host. Waypipe at least serializes the buffers "intelligently" (compression, damage-regions etc.). AFAIK X11 will just push the raw bytes through the vsock. https://github.com/AsahiLinux/muvm does something more intelligent with DRM native context; maybe this could be adapted. anyway, I'll address the merge conflicts soon. |
Adds a `--waypipe` machine flag that bridges a guest waypipe vsock port to a host AF_UNIX socket in the VM data dir, so guest GUI applications render on the host Wayland compositor. No X11, no SSH server, and no TCP port forward: forwarding rides the same vsock mechanism as --cuda. The topology is outbound (listen=false, like the CUDA/SSH/DNS bridges): the guest runs `waypipe server --vsock` and connects out to host CID 2 on port 7001 (ports::WAYPIPE); libkrun bridges that port to `<vmdir>/waypipe.sock`, where the user runs a listening `waypipe client` next to the host compositor. The host socket is created lazily on the first guest connect, so the client must be started first. Wiring mirrors the existing docker_socket bool feature end to end: - ports::WAYPIPE constant (smolvm-protocol) - WaypipeService in the vsock-service registry (listen=false) - LaunchFeatures.waypipe / LaunchConfig.waypipe_socket - BootConfig.waypipe, written from features in the manager - internal_boot derives waypipe.sock and clears any stale node - --waypipe on `machine create`/`run`, persisted to the VM record and merged like the other capability flags; a hint is printed on run - Smolfile `waypipe` field Verified on ubuntu:24.04 (virtio-net): weston-flower rendered on the host compositor, with both a guest server-conn and a host client-conn worker for waypipe.sock confirming the full chain. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a `--x11` machine flag that bridges guest X11 applications to the host X server, so they render on the host desktop. X11 is network-transparent by design, so the bytes pass through unmodified -- no server/client pair and no forwarding protocol, just a byte pipe over vsock. The topology is outbound (listen=false): the guest X client connects out to host CID 2 on port 7002 (ports::X11); libkrun bridges that port straight to the host X server socket. Unlike the other vsock endpoints, the host socket is the LIVE X server socket (not smolvm-owned): internal_boot resolves it from the host $DISPLAY via host_x11_socket() (`:N`/`:N.S` -> /tmp/.X11-unix/XN; skipped for TCP displays), and it is never unlinked. If $DISPLAY is unset or the socket is missing, the bridge is disabled with a warning rather than aborting the boot. The guest side is fully automated -- no manual socat. The guest agent runs a bridge (crates/smolvm-agent/src/x11.rs) that binds a local display socket (/tmp/.X11-unix/X10) and relays each connection out to the vsock port, and exports DISPLAY=:10 for the workload. This mirrors the SSH agent bridge end to end: activated by the SMOLVM_X11=1 guest_env signal from X11Service, started in the agent boot sequence, and injected into container specs and exec/run env (inject_into_container / inject_into_env) so DISPLAY reaches both bare-VM and image workloads. Display :10 is hardcoded for now. The relay uses the long-lived, half-close style of the Docker bridge, since X11 connections stay open. The exec/run env injection also runs on the interactive `crun exec` join path (spawn_exec_in_container): a `-it` join builds a fresh process env rather than inheriting the container's, so without this SSH_AUTH_SOCK and DISPLAY were silently dropped for interactive sessions on a persistent machine. Host-side wiring: ports::X11, X11Service in the vsock registry, LaunchFeatures/LaunchConfig/BootConfig, manager, internal_boot, CLI flag + hint + VM record, Smolfile. Verified on ubuntu:24.04 (virtio-net): xeyes rendered on the host X server with no socat, DISPLAY set automatically, on both the ephemeral `run` path and the interactive `-it` join into a persistent machine. Note: a plain byte pipe cannot pass SCM_RIGHTS ancillary fds, so MIT-SHM and DRI3 fall back to wire-image transport (correct, just slower). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tics Before, --waypipe only wired the vsock transport: the user had to run `waypipe server` per app in the guest, set WAYLAND_DISPLAY, and run a host `waypipe client` by hand. This makes a plain `machine exec -- gui-app` just work, wires up both ends automatically, and reports clearly when it cannot. Guest daemon (crates/smolvm-agent/src/waypipe.rs): - Run `waypipe server` in --display daemon mode inside the workload container, so one daemon serves every app (like the X11 display socket) and WAYLAND_DISPLAY is exported for the workload. The daemon MUST run in the container, not the agent: the agent rootfs is musl and cannot exec a glibc waypipe. Started via a detached `crun exec` into the long-lived keep-alive container; persists across execs; idempotent (socket-presence check) and self-healing (retried each exec until waypipe is present). - Binary source is selectable via --waypipe[=host|container|PATH]: `host` (default) shares the host binary over virtiofs so guest server and host client are the same build (no wire drift, image needs no waypipe); `container` uses the image's own waypipe; a path shares that binary. Host client (src/vm/waypipe.rs, src/cli/internal_boot.rs): - Start the matching host `waypipe client` automatically on the host next to the compositor (Linux only; macOS/Windows have no Wayland). Held for the VM's lifetime and armed with PR_SET_PDEATHSIG so it dies with the boot process (which exits via _exit, skipping Drop). Fail-soft, like the X11 bridge: if $WAYLAND_DISPLAY is unset or no waypipe is on PATH, it warns and skips, and the user can run one by hand. Diagnostics - forwarding never fails silently: - --waypipe without an --image is rejected up front on run and create: a bare VM has no container to host the daemon, so it could never work. - start_daemon_in_container returns a DaemonStartOutcome, classified by polling for the display socket after the start exec (the backgrounded daemon makes the exec's own exit status uninformative) and probing for the binary. `machine exec -- cmd` prepends a one-line warning to the command's stderr (stdout untouched); an interactive `-it` shell prints it at the top of the session (CRLF for the raw TTY). Missing waypipe says how to install it and that it starts on the next command; a present-but-broken waypipe (e.g. host/guest glibc mismatch in host mode) suggests --waypipe=container. Only a running daemon is silent. Config threading: --waypipe / waypipe_bin flow through the Smolfile, VM record, CreateVmParams/DefaultVmOverrides, LaunchFeatures, BootConfig, and LaunchConfig so persistent machines re-resolve the binary source on start. Verified on ubuntu:26.04: host and container modes both render weston-terminal on the host compositor with no manual steps; the host client is spawned automatically and dies on stop/delete (no leaks); bare run/create --waypipe error clearly; missing waypipe warns with install guidance (interactive and non-interactive) and comes up on the next exec after install; a broken waypipe surfaces the failure. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
client.rs called crate::host::fnv64, but the host module is gated behind the "host" feature. The client module always compiles, so building smolvm-cuda without "host" (the standalone guest/agent-rootfs build via smolvm-cuda-guest, which sets default-features = false) failed with "cannot find host in crate". This broke `cargo make agent-rootfs`. Move the pure fnv64 helper out of the feature-gated host module into the always-compiled crate root, and update every caller (client, host, and the host binary's cuda_daemon) to crate::fnv64 / smolvm_cuda::fnv64. No behavior change; feature-unified host builds already saw the same fn. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pumphaus
force-pushed
the
feature/waypipe-vsock
branch
from
July 20, 2026 06:51
1f4583a to
0a3c344
Compare
Author
|
Rebased. Also needed a small fix on top to make agent-rootfs compile again. I've had to download 1.2 GB (!!) for the few commits on main since my fork; what's been going there? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds two machine flags for running guest GUI apps on the host desktop, both fully automated end to end — no
socat, no manualwaypipeinvocations, no per-app setup. Both ride the existing vsock mechanism (same as--cuda), so no networking is required for forwarding itself.--x11— bridge guest X11 apps straight to the host X server.--waypipe— forward guest Wayland apps to the host compositor via waypipe, with the guest daemon and the host client started for you.--x11X11 is network-transparent by design, so bytes pass through unmodified — just a byte pipe over vsock (guest connects out to host CID 2, port 7002).
internal_bootresolves the live host X server socket from$DISPLAY(:N/:N.S->/tmp/.X11-unix/XN; TCP displays skipped). The guest agent binds a local display socket, relays each connection out, and exportsDISPLAY=:10— for bare-VM and image workloads, and on the interactive-itjoin path. If$DISPLAYis unset or the socket is missing, the bridge disables with a warning rather than aborting boot.Limitation: a plain byte pipe can't pass
SCM_RIGHTSfds, so MIT-SHM and DRI3 fall back to wire-image transport (correct, just slower). For per-window integration and correct fd/GPU handling, prefer--waypipe.--waypipeWayland can't be byte-relayed — every frame/keymap/clipboard rides as an
SCM_RIGHTSfd — so this runs waypipe, which terminates the protocol on each side.waypipe serverin--displaymode, one daemon serving every app (like the X11 display socket),WAYLAND_DISPLAYexported automatically. The daemon runs inside the workload container, not the agent: the agent rootfs is musl and can't exec a glibc waypipe. Started via a detachedcrun execinto the keep-alive container; idempotent and self-healing (retried each exec until waypipe is present).waypipe clientis started automatically on the host next to the compositor (Linux only). Tied to the VM lifetime viaPR_SET_PDEATHSIGso it dies with the VM. Fail-soft: if$WAYLAND_DISPLAYis unset or nowaypipeon PATH, it warns, skips, and you can run one by hand.--waypipe[=host|container|PATH].host(default) shares the host binary over virtiofs (same build on both ends, no wire drift, image needs no waypipe);containeruses the image's own; a path shares that binary.Diagnostics — forwarding never fails silently
--waypipewithout--imageis rejected up front (a bare VM has no container to host the daemon).machine exec -- cmdprepends a one-line warning to stderr (stdout untouched); an interactive-itshell prints it at the top of the session. Missing waypipe says how to install it (and that it starts on the next command); a present-but-broken waypipe (e.g. host/guest glibc mismatch inhostmode) suggests--waypipe=container. Only a running daemon is silent.Verification (ubuntu:24.04 / 26.04)
--x11:xeyesrendered on the host X server,DISPLAYauto-set, on both ephemeralrunand the interactive-itjoin.--waypipe:weston-terminal/weston-flowerrendered on the host compositor in both host and container modes; host client spawned automatically and dies on stop/delete (no leaks); bare-VM rejection, missing-waypipe warning + install-then-works cycle, and broken-waypipe failure all confirmed.Commits
Add --waypipe: forward guest Wayland apps to the host over vsockAdd --x11: bridge guest X11 apps to the host X server over vsockAutomate --waypipe end to end: guest daemon, host client, and diagnostics🤖 Generated with Claude Code