Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ you can "cook" your containerized application into a portable single-file app, m
| **Internal networking** (services reach each other via `127.0.0.1:<port>`) | ✅ | ✅ | ✅ verified |
| **Host port mapping — TCP** (`"host:guest"`, host≠guest proxied) | ✅ | ✅ verified | ✅ verified — the vz backend relays `127.0.0.1:host → <vm_ip>:guest` itself (the cross-platform runtime proxy is skipped on macOS) |
| **Host port mapping — UDP** | ✅ | ✅ verified — Chefer relays via the VM IP + an in-VM eth0→loopback bridge (WSL2's own forwarding is TCP-only) | ✅ verified (same VM-IP relay + in-VM bridge) |
| **GUI services** | ✅ X11 / Wayland socket passthrough | ✅ WSL2 via WSLg ([gui-demo](examples/gui-demo)); **✅ WHP (no WSL)** — bundle-embedded `cage` compositor on a virtio-gpu scanout shown in a **resizable** native window, with keyboard/mouse + bidirectional clipboard (text validated bidirectionally on real hardware; PNG images via the `PNG` format / `CF_DIB`, host read/convert path verified via `--clip-selftest`; resizing the window scales the picture — the resolution-change signal reaches the guest driver, verified on real hardware, but the `cage` kiosk compositor keeps its native mode, so there's no true guest re-modeset) | ✅ validated on real Apple Silicon — `VZVirtualMachineView` window + the same in-guest `cage` path as WHP (compositor up 8/8 boots); clipboard verified (text both ways; PNG guest→host byte-exact, host→guest wire-confirmed). Resolution: the **initial** guest mode matches the window's *pixel* size (Retina 2× backing; `CHEFER_VZ_GUI_SIZE` honored — verified 1280×800→2560×1600 and 1024×600→2048×1200), but **live drag-resize only scales the picture** — same `cage` kiosk-compositor limitation as WHP (the compositor doesn't follow mode changes after startup; verified on real hardware: guest stayed 2560×1600 across window drags). Interactive HID / close-window remain hand-checks (`scripts/vz-smoke.sh --gui`). Caveat: the clipboard channel is sometimes silently delayed — suspected macOS Local Network privacy gating of the ad-hoc-signed helper |
| **GUI services** | ✅ X11 / Wayland socket passthrough | ✅ WSL2 via WSLg ([gui-demo](examples/gui-demo)); **✅ WHP (no WSL)** — bundle-embedded `cage` compositor on a virtio-gpu scanout shown in a **resizable** native window, with keyboard/mouse + bidirectional clipboard (text validated bidirectionally on real hardware; PNG images via the `PNG` format / `CF_DIB`, host read/convert path verified via `--clip-selftest`; resizing the window now triggers a **true guest re-modeset**: the in-guest resize watcher follows the virtio-gpu display change (drm hotplug → re-probe → wlr-output-management; needs the cage ≥ 0.2.0 GUI overlay) — the chain is verified end-to-end on the vz backend, WHP re-validation pending; older overlays fall back to scaling the picture) | ✅ validated on real Apple Silicon — `VZVirtualMachineView` window + the same in-guest `cage` path as WHP (compositor up 8/8 boots); clipboard verified (text both ways; PNG guest→host byte-exact, host→guest wire-confirmed). Resolution: the **initial** guest mode matches the window's *pixel* size (Retina 2× backing; `CHEFER_VZ_GUI_SIZE` honored — verified 1280×800→2560×1600 and 1024×600→2048×1200), and live resize **re-modesets the guest** with dynamic resolution opted in (`CHEFER_VZ_DYNAMIC_RESOLUTION=1`, macOS 14+, cage ≥ 0.2.0 overlay — verified on real hardware: `xdpyinfo` followed 2560×1600 → 2200×1400); the default stays view-scaling (Retina-pixel modes halve Linux UI sizes; no HiDPI output scale yet). Interactive HID / close-window remain hand-checks (`scripts/vz-smoke.sh --gui`). Caveat: the clipboard channel is sometimes silently delayed — suspected macOS Local Network privacy gating of the ad-hoc-signed helper |
| **`crash: fail_fast`** (any non-zero exit tears down the app, code propagated) | ✅ | ✅ verified | ✅ verified (exit 7 propagated; interface-service exit-0 teardown too) |
| **Data-dir migration** (`old_names`) | ✅ | ✅ | ✅ |
| **Official chown/gosu images** (redis, postgres, …) | ✅ as root (no userns); ✅ rootless via `/etc/subuid` + `newuidmap` (falls back to single-uid without them) | ✅ verified — distro runs as real root, runs official redis/postgres as-is | ✅ (same real-root guest path as WHP) |
Expand Down Expand Up @@ -264,5 +264,5 @@ Honest list of what doesn't work (yet):
- [x] **`depends_on` health checks** (wait-until-ready), not just start order — a service with `healthcheck` is polled inside its container namespaces until healthy before later services start; services without one are considered ready once spawned.
- [x] **Rootless Linux support for chown/gosu images** via `newuidmap` + `/etc/subuid` delegation — when the host has the `uidmap` tools and subuid/subgid ranges for the user (most distros ship both), rootless services get a full uid-range mapping (rootless-Podman-style) and images that `chown`/`gosu` to a dedicated uid (official redis/postgres) run as-is; verified end-to-end with the official `redis` image as a non-root user. Falls back to the old single-uid map when delegation is unavailable. (The root backends — WSL2 / macOS VM / native-root — never needed this.)
- [x] **macOS VZ boot validated on real Apple Silicon** *(2026-07, Apple Silicon / macOS 26)* — `scripts/vz-smoke.sh` fully green on bare metal: VM boot + virtiofs + guest-agent, exit-code propagation, service supervision, host→guest **TCP and UDP** port relays, multi-service `depends_on` + in-app `127.0.0.1` networking, `persist_path` write-back across restarts, and default-`bridge` outbound (pasta → VZ NAT, DNS included). Real-hardware fixes that came out of it: tar-mode restore on extraction (bundled agents lost their exec bit), an arm64-appliance initramfs that shipped an x86-64 busybox (cross-arch `busybox-static` now fetched at build), the runtime port proxy double-binding against the vz relay, and Docker-parity `/etc/resolv.conf` injection into service containers (without it, bridge outbound had connectivity but no DNS). Note: VZ is virtualization, not emulation — guest arch must match host (Apple Silicon → `linux/arm64`, Intel → `linux/amd64`). The backend still requires `CHEFER_VZ_EXPERIMENTAL=1` — flipping it default-on is a separate decision.
- [x] **GUI apps on macOS** — *validated on real Apple Silicon (2026-07).* The in-guest half (kiosk `cage` + Xwayland on a virtio-gpu scanout) is shared with WHP; `chefer-vz-helper --gui` shows the VM in an AppKit window via `VZVirtualMachineView` (1280×800 default, `CHEFER_VZ_GUI_SIZE=WxH` override) with USB keyboard/absolute-pointer devices. Real-hardware results: the compositor came up **8/8 GUI boots** (the `WLR_LIBINPUT_NO_DEVICES` boot-race fix holds on VZ); X11 clients connect and render; the host↔guest **clipboard** works (text verified both ways; PNG guest→host byte-exact on `NSPasteboard`, host→guest wire-confirmed). Resolution: `automaticallyReconfiguresDisplay` (macOS 14+) does push the window's *pixel* size to the guest — the **initial** mode matches the window's Retina backing exactly (a 1280×800-point window yields a 2560×1600 guest mode; `CHEFER_VZ_GUI_SIZE=1024x600` yields 2048×1200) — but **live drag-resize only scales the picture**, same as WHP: the `cage` kiosk compositor doesn't follow mode changes after startup (verified on real hardware — guest stayed 2560×1600 across drags; the fix would be guest-side, e.g. teaching the compositor to re-modeset on virtio-gpu hotplug). Interactive checks that still need a human hand (`scripts/vz-smoke.sh --gui` checklist): typing/clicking into the guest, close-window-ends-app. Known caveat: the clipboard channel (helper → `<vm_ip>:55381` over VZ NAT) sometimes sits in a silent `.waiting` state for up to a minute or never connects — suspected macOS Local Network privacy gating of the ad-hoc-signed helper; needs a stable signing identity and a visible hint in the helper. Design in [docs/DESIGN.md](docs/DESIGN.md).
- [x] **GUI apps on macOS** — *validated on real Apple Silicon (2026-07).* The in-guest half (kiosk `cage` + Xwayland on a virtio-gpu scanout) is shared with WHP; `chefer-vz-helper --gui` shows the VM in an AppKit window via `VZVirtualMachineView` (1280×800 default, `CHEFER_VZ_GUI_SIZE=WxH` override) with USB keyboard/absolute-pointer devices. Real-hardware results: the compositor came up **8/8 GUI boots** (the `WLR_LIBINPUT_NO_DEVICES` boot-race fix holds on VZ); X11 clients connect and render; the host↔guest **clipboard** works (text verified both ways; PNG guest→host byte-exact on `NSPasteboard`, host→guest wire-confirmed). Resolution: `automaticallyReconfiguresDisplay` (macOS 14+) does push the window's *pixel* size to the guest — the **initial** mode matches the window's Retina backing exactly (a 1280×800-point window yields a 2560×1600 guest mode; `CHEFER_VZ_GUI_SIZE=1024x600` yields 2048×1200) — and live resize now **re-modesets the guest**: `cage` doesn't follow mode changes by itself, so the guest agent's resize watcher listens for the drm hotplug, re-probes the new preferred mode, and applies it over wlr-output-management (needs the cage ≥ 0.2.0 overlay from the current `build-gui-overlay.sh`; verified end-to-end on real hardware with `CHEFER_VZ_DYNAMIC_RESOLUTION=1`: `xdpyinfo` followed 2560×1600 → 2200×1400). The **default** stays view-scaling — Retina-pixel modes halve Linux UI sizes until the compositor gains HiDPI output scale; the opt-in flips it to true dynamic resolution. Interactive checks that still need a human hand (`scripts/vz-smoke.sh --gui` checklist): typing/clicking into the guest, close-window-ends-app. Known caveat: the clipboard channel (helper → `<vm_ip>:55381` over VZ NAT) sometimes sits in a silent `.waiting` state for up to a minute or never connects — suspected macOS Local Network privacy gating of the ad-hoc-signed helper; needs a stable signing identity and a visible hint in the helper. Design in [docs/DESIGN.md](docs/DESIGN.md).
- [x] **GPU access / CUDA compute** — *opt-in `gpu: true`; validated end-to-end on both WSL2 and native NVIDIA Linux.* Set `gpu: true` on a service and Chefer binds the host's render/compute device nodes into that service's container — on **native Linux** `/dev/nvidia*` + `/dev/dri`; on the **Windows WSL2 backend** `/dev/dxg` + a bind of the host `/usr/lib/wsl/lib` **and `/usr/lib/wsl/drivers`** (its host-version-matched `libdxcore`/`libcuda`/… ) → CUDA/DirectML/OpenCL/Vulkan work with nothing extra in the image. The same passthrough also covers **NVENC/NVDEC video** and OpenCL/ROCm compute; **GL/Vulkan *rendering* does not work in a headless container** (see the native note below). **Validated end-to-end on real WSL2** (GeForce GT 1030) — the container sees `/dev/dxg` + `/usr/lib/wsl/lib` + `/usr/lib/wsl/drivers`, runs `nvidia-smi` with live telemetry, and executes a `nvcc`-built CUDA `vectorAdd` on the GPU (`sm_61`, 1048576 elems, 0 mismatches); binding `/usr/lib/wsl/lib` alone is not enough — its `libcuda`/`libnvidia-ml` are loader shims that dlopen the real driver from `/usr/lib/wsl/drivers`, so both must be bound (else `nvidia-smi` reports "Driver Not Loaded"). `/dev/dri` is bound when present (absent on some WSL kernels; CUDA uses `/dev/dxg`). It is a **clear startup error** on the non-WSL Windows (WHP) micro-VM and the macOS VM — a bare WHP/VZ VM can't tap Microsoft/Apple GPU paravirtualization, so GPU compute stays **native-Linux + WSL2 only**. **Native Linux** — host-driver-version-matched userspace lib injection (à la the NVIDIA Container Toolkit) reads the driver version from `/proc/driver/nvidia/version` and binds the matching NVIDIA driver libs (`libcuda`/`libnvidia-*`/`libnvcuvid`/`libnvoptix`/GL, matching `nvidia-container-cli list`) into `/run/chefer-nvidia` (versioned + real DT_SONAME + `.so`) plus `LD_LIBRARY_PATH`. **Validated on a GeForce RTX 4070** (driver 570.211.01): a `nvcc`-built CUDA vectorAdd from a stock `nvidia/cuda` image that ships no libcuda ran on the GPU (`sm_89`, 1048576 elems, 0 mismatches), and `ffmpeg -c:v h264_nvenc` encoded in-container (NVENC, exit 0). **OpenGL/Vulkan *rendering* does not work in a headless container**, though: NVIDIA's Vulkan ICD (`libGLX_nvidia` → `libnvidia-glcore`) needs a real graphics/X environment (it references Xorg's `ErrorF` and glibc-2.34-removed `__malloc_hook`), so it falls back to `llvmpipe` — and `docker --gpus all` with `NVIDIA_DRIVER_CAPABILITIES=all` (the nvidia-container-toolkit reference) fails identically on the same headless host, so this is a host/driver limitation, not a Chefer one (Chefer's GUI path renders via X11/Wayland socket passthrough + software rasterization instead). No path to Apple NPU / CoreML from a Linux guest.
12 changes: 9 additions & 3 deletions crates/guest-agent/src/clipboard.rs
Original file line number Diff line number Diff line change
Expand Up @@ -332,9 +332,15 @@ fn wl_copy_kind(kind: u8, data: &[u8]) {
}
}

/// 在 PATH 找執行檔(overlay 解壓後的 /usr/bin 等)。
fn which(bin: &str) -> Option<std::path::PathBuf> {
let path = std::env::var_os("PATH")?;
/// 在 PATH 找執行檔(overlay 解壓後的 /usr/bin 等)。resize.rs 亦共用。
///
/// PATH 未設時退回標準路徑:appliance init 舊版沒 export PATH(kernel 給 PID 1 的環境
/// 本來就沒有),guest-agent 會拿不到——別因此把剪貼簿/動態解析度整組靜默停用。
/// 執行時 `Command::new(裸名)` 的 execvp 在無 PATH 下也用類似的預設路徑,行為一致。
pub(crate) fn which(bin: &str) -> Option<std::path::PathBuf> {
let path = std::env::var_os("PATH")
.filter(|p| !p.is_empty())
.unwrap_or_else(|| "/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin".into());
for dir in std::env::split_paths(&path) {
let cand = dir.join(bin);
if cand.is_file() {
Expand Down
15 changes: 12 additions & 3 deletions crates/guest-agent/src/gui.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@
//! `$XDG_RUNTIME_DIR/wayland-0`(與 Xwayland 的 `/tmp/.X11-unix/X0`)出現。
//! 4. 設定 `XDG_RUNTIME_DIR`/`WAYLAND_DISPLAY`/`DISPLAY` 供 exec.rs 既有的
//! GUI socket bind 邏輯把 socket 掛進服務容器。
//! 5. 啟動 host↔guest 剪貼簿同步(clipboard.rs)與動態解析度跟隨(resize.rs——
//! host 調整視窗 → guest re-modeset,cage 本身不跟隨線上模式變更)。
//!
//! 觸發條件(`maybe_start`):appliance init 設 `CHEFER_VM_GUI=1` **且** manifest
//! 有 gui/both 服務。overlay 缺失 → 硬錯誤(依契約不得無聲黑屏)。
Expand All @@ -29,7 +31,7 @@ use nix::mount::{MsFlags, mount};
/// 記一行 GUI 診斷。同時走 stderr(原生 Linux / 有接 console 的後端可見)與 **`/dev/kmsg`**
/// ——WHP 的序列 console 在 `quiet` 下不顯示 guest-agent 自己的 stderr,但高優先序 kmsg 會
/// 出現在 console(同 appliance init 的 `report()`),故 WHP GUI 的啟動診斷才看得到。best-effort。
fn note(msg: &str) {
pub(crate) fn note(msg: &str) {
eprintln!("[guest-agent] gui: {msg}");
if let Ok(mut f) = std::fs::OpenOptions::new().write(true).open("/dev/kmsg") {
use std::io::Write as _;
Expand All @@ -47,18 +49,21 @@ const GUI_RUNTIME_DIR: &str = "/run/chefer-gui";
/// 等 compositor socket 出現的上限。llvmpipe 首次啟動慢,放寬一點。
const SOCKET_WAIT: Duration = Duration::from_secs(20);

/// 存活中的 GUI 環境;Drop 時收掉 clipboard / cage / seatd / udevd。
/// 存活中的 GUI 環境;Drop 時收掉 resize/clipboard / cage / seatd / udevd。
pub struct GuiSession {
cage: Child,
seatd: Option<Child>,
udevd: Option<Child>,
/// host↔guest 剪貼簿同步(有 cmdline token 且 wl-clipboard 可用時;見 clipboard.rs)。
clipboard: Option<crate::clipboard::ClipboardSync>,
/// 動態解析度跟隨(wlr-randr 可用時;見 resize.rs)。
resize: Option<crate::resize::ResizeWatcher>,
}

impl Drop for GuiSession {
fn drop(&mut self) {
// 先收剪貼簿同步(背景執行緒),再收 compositor。
// 先收背景執行緒(解析度跟隨、剪貼簿同步),再收 compositor。
self.resize.take();
self.clipboard.take();
let _ = self.cage.kill();
let _ = self.cage.wait();
Expand Down Expand Up @@ -111,6 +116,7 @@ pub fn maybe_start(bundle_dir: &Path, manifest: &Manifest) -> Result<Option<GuiS
seatd,
udevd,
clipboard: None,
resize: None,
};

let wayland = PathBuf::from(GUI_RUNTIME_DIR).join("wayland-0");
Expand All @@ -137,6 +143,9 @@ pub fn maybe_start(bundle_dir: &Path, manifest: &Manifest) -> Result<Option<GuiS
// compositor 就緒後起剪貼簿同步(env 已設好,wl-clipboard 可接 cage)。無 cmdline
// token(host 未啟用剪貼簿)或 wl-clipboard 缺失時回 None,不致命。
session.clipboard = crate::clipboard::maybe_start();
// 動態解析度跟隨(host 調整視窗 → guest re-modeset;見 resize.rs)。wlr-randr 缺失
//(舊 overlay)時回 None,不致命——退回「host 端拉伸顯示」的舊行為。
session.resize = crate::resize::maybe_start();
Ok(Some(session))
}

Expand Down
2 changes: 2 additions & 0 deletions crates/guest-agent/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ pub mod health;
#[cfg(target_os = "linux")]
pub mod netns;
#[cfg(target_os = "linux")]
pub mod resize;
#[cfg(target_os = "linux")]
pub mod subid;
#[cfg(target_os = "linux")]
pub mod supervisor;
Expand Down
Loading
Loading