Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,5 @@
- **vz-smoke.sh 的 pkill 治標可升級為斷言**:helper stdin-EOF 自我了結(vz.rs + vz-helper/main.swift)合併後,`scripts/vz-smoke.sh` 收尾的 `pkill -f chefer-vz-helper`(目前在主 checkout 未 commit 的變更中)可改成「等 ~10s 斷言無 chefer-vz-helper 殘留」,讓實機一鍵驗證直接覆蓋這個修復。
- **runtime 單發 SIGINT 下 helper 收攤有 ~5s 延遲**:runtime 的 ctrlc handler 等 5 秒才 `exit(130)`,stdin EOF 要到行程死亡才發生(實測 SIGINT→helper 收攤約 6s;SIGTERM/SIGKILL 即時)。若要即時收攤,可讓 vz 後端把 helper stdin 寫端交給訊號處理路徑、收訊號時主動關閉。屬優化,非正確性問題。
- **whp 防孤兒修復待實機驗證**:Job Object(KILL_ON_JOB_CLOSE)+ helper stdin-EOF 自我了結(`crates/vmm-backend/src/whp.rs` + `crates/whp-helper/src/main.rs`)已完成、機制各有 CI 單元測試,但「單殺 runtime 後 helper/VM 不殘留」的端到端行為需實體 Windows(WHP)驗證:以 `CHEFER_BACKEND=whp` 跑一個 bundle → `taskkill /F /PID <runtime pid>`(TerminateProcess,孤兒化的可靠重現法;原始問題本身也尚未在實機重現過)→ 確認 `chefer-whp-helper` 行程數秒內消失、無殘留。通過後刪除本項。
- **vz bridge 出網(pasta→apk)於 2026-07-12 實測不通,疑點待查**:HiDPI e2e 時 guest 內 `apk add`(`network: bridge`)連續 10+ 次「temporary error」(`dl-cdn.alpinelinux.org` main/community 皆然),同時 host 直連同 URL HTTP 200;pasta 有起、netns 正常、resolv.conf 注入路徑(exec.rs)在。可能是本次網路環境(Wi-Fi/DNS)因素,也可能是 container-DNS 改動(PR #127,僅 WHP 實機驗證)合併後 vz 側回歸——f9f705b 驗證 vz bridge 出網是在該合併**之前**。建議:在正常網路下重跑 `vz-smoke.sh --gui`(xclock 的 apk 步驟即覆蓋),通則屬環境因素刪本項;仍失敗則進 guest 看 `/etc/resolv.conf` 與 pasta DNS 轉發。
- **branch protection 的必要檢查名稱失效,每個 PR 都得 admin bypass**:main 的 required status checks 含一條 `whp-helper (windows compile-check)`(無後綴),但該 CI job 從建立起就是 matrix(`.github/workflows/ci.yml`),實際回報名稱是 `whp-helper (windows compile-check) (x86_64-pc-windows-msvc)` 與 `(aarch64-pc-windows-msvc)`——這條 context 永遠不會被滿足,所有 PR 的 merge state 都是 BLOCKED,只能靠 owner `--admin`/網頁勾「bypass」合併(2026-07-11 PR #126 實測確認)。修法(需 repo 管理權限,Settings → Branches → main 的 required status checks,或 `gh api` PATCH `branches/main/protection/required_status_checks`):把無後綴那條換成上述兩條帶 target 後綴的。修好後刪除本項。
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ you can "cook" your containerized application into a portable single-file app, m
| **Internal networking** (services reach each other via `127.0.0.1:<port>`) | ✅ | ✅ | ✅ verified |
| **Host port mapping — TCP** (`"host:guest"`, host≠guest proxied) | ✅ | ✅ verified | ✅ verified — the vz backend relays `127.0.0.1:host → <vm_ip>:guest` itself (the cross-platform runtime proxy is skipped on macOS) |
| **Host port mapping — UDP** | ✅ | ✅ verified — Chefer relays via the VM IP + an in-VM eth0→loopback bridge (WSL2's own forwarding is TCP-only) | ✅ verified (same VM-IP relay + in-VM bridge) |
| **GUI services** | ✅ X11 / Wayland socket passthrough | ✅ WSL2 via WSLg ([gui-demo](examples/gui-demo)); **✅ WHP (no WSL)** — bundle-embedded `cage` compositor on a virtio-gpu scanout shown in a **resizable** native window, with keyboard/mouse + bidirectional clipboard (text validated bidirectionally on real hardware; PNG images via the `PNG` format / `CF_DIB`, host read/convert path verified via `--clip-selftest`; resizing the window now triggers a **true guest re-modeset**: the in-guest resize watcher follows the virtio-gpu display change (drm hotplug → re-probe → wlr-output-management; needs the cage ≥ 0.2.0 GUI overlay) — the chain is verified end-to-end on the vz backend, WHP re-validation pending; older overlays fall back to scaling the picture) | ✅ validated on real Apple Silicon — `VZVirtualMachineView` window + the same in-guest `cage` path as WHP (compositor up 8/8 boots); clipboard verified (text both ways; PNG guest→host byte-exact, host→guest wire-confirmed). Resolution: the **initial** guest mode matches the window's *pixel* size (Retina 2× backing; `CHEFER_VZ_GUI_SIZE` honored — verified 1280×800→2560×1600 and 1024×600→2048×1200), and live resize **re-modesets the guest** with dynamic resolution opted in (`CHEFER_VZ_DYNAMIC_RESOLUTION=1`, macOS 14+, cage ≥ 0.2.0 overlay — verified on real hardware: `xdpyinfo` followed 2560×1600 → 2200×1400); the default stays view-scaling (Retina-pixel modes halve Linux UI sizes; no HiDPI output scale yet). Interactive HID / close-window remain hand-checks (`scripts/vz-smoke.sh --gui`). Caveat: the clipboard channel is sometimes silently delayed — suspected macOS Local Network privacy gating of the ad-hoc-signed helper |
| **GUI services** | ✅ X11 / Wayland socket passthrough | ✅ WSL2 via WSLg ([gui-demo](examples/gui-demo)); **✅ WHP (no WSL)** — bundle-embedded `cage` compositor on a virtio-gpu scanout shown in a **resizable** native window, with keyboard/mouse + bidirectional clipboard (text validated bidirectionally on real hardware; PNG images via the `PNG` format / `CF_DIB`, host read/convert path verified via `--clip-selftest`; resizing the window now triggers a **true guest re-modeset**: the in-guest resize watcher follows the virtio-gpu display change (drm hotplug → re-probe → wlr-output-management; needs the cage ≥ 0.2.0 GUI overlay) — the chain is verified end-to-end on the vz backend, WHP re-validation pending; older overlays fall back to scaling the picture) | ✅ validated on real Apple Silicon — `VZVirtualMachineView` window + the same in-guest `cage` path as WHP (compositor up 8/8 boots); clipboard verified (text both ways; PNG guest→host byte-exact, host→guest wire-confirmed). Resolution: the **initial** guest mode matches the window's *pixel* size (Retina 2× backing; `CHEFER_VZ_GUI_SIZE` honored — verified 1280×800→2560×1600 and 1024×600→2048×1200), and live resize **re-modesets the guest** with dynamic resolution opted in (`CHEFER_VZ_DYNAMIC_RESOLUTION=1`, macOS 14+, cage ≥ 0.2.0 overlay — verified on real hardware: `xdpyinfo` followed 2560×1600 → 2200×1400); true dynamic resolution is now the **default** on macOS 14+ — the guest re-modesets on window resize and applies the host's HiDPI output scale (pushed via kernel cmdline `chefer.gui_scale`) so cursor/UI sizes stay correct on Retina; verified on real hardware including hand-checks (cursor size, pointer mapping, upscale quality; `xdpyinfo` reports logical points 1280×800 → 1100×700 across a live resize). `CHEFER_VZ_DYNAMIC_RESOLUTION=0` (or macOS 13) falls back to view-scaling. Interactive HID / close-window remain hand-checks (`scripts/vz-smoke.sh --gui`). Caveat: the clipboard channel is sometimes silently delayed — suspected macOS Local Network privacy gating of the ad-hoc-signed helper |
| **`crash: fail_fast`** (any non-zero exit tears down the app, code propagated) | ✅ | ✅ verified | ✅ verified (exit 7 propagated; interface-service exit-0 teardown too) |
| **Data-dir migration** (`old_names`) | ✅ | ✅ | ✅ |
| **Official chown/gosu images** (redis, postgres, …) | ✅ as root (no userns); ✅ rootless via `/etc/subuid` + `newuidmap` (falls back to single-uid without them) | ✅ verified — distro runs as real root, runs official redis/postgres as-is | ✅ (same real-root guest path as WHP) |
Expand Down Expand Up @@ -264,5 +264,5 @@ Honest list of what doesn't work (yet):
- [x] **`depends_on` health checks** (wait-until-ready), not just start order — a service with `healthcheck` is polled inside its container namespaces until healthy before later services start; services without one are considered ready once spawned.
- [x] **Rootless Linux support for chown/gosu images** via `newuidmap` + `/etc/subuid` delegation — when the host has the `uidmap` tools and subuid/subgid ranges for the user (most distros ship both), rootless services get a full uid-range mapping (rootless-Podman-style) and images that `chown`/`gosu` to a dedicated uid (official redis/postgres) run as-is; verified end-to-end with the official `redis` image as a non-root user. Falls back to the old single-uid map when delegation is unavailable. (The root backends — WSL2 / macOS VM / native-root — never needed this.)
- [x] **macOS VZ boot validated on real Apple Silicon** *(2026-07, Apple Silicon / macOS 26)* — `scripts/vz-smoke.sh` fully green on bare metal: VM boot + virtiofs + guest-agent, exit-code propagation, service supervision, host→guest **TCP and UDP** port relays, multi-service `depends_on` + in-app `127.0.0.1` networking, `persist_path` write-back across restarts, and default-`bridge` outbound (pasta → VZ NAT, DNS included). Real-hardware fixes that came out of it: tar-mode restore on extraction (bundled agents lost their exec bit), an arm64-appliance initramfs that shipped an x86-64 busybox (cross-arch `busybox-static` now fetched at build), the runtime port proxy double-binding against the vz relay, and Docker-parity `/etc/resolv.conf` injection into service containers (without it, bridge outbound had connectivity but no DNS). Note: VZ is virtualization, not emulation — guest arch must match host (Apple Silicon → `linux/arm64`, Intel → `linux/amd64`). The backend still requires `CHEFER_VZ_EXPERIMENTAL=1` — flipping it default-on is a separate decision.
- [x] **GUI apps on macOS** — *validated on real Apple Silicon (2026-07).* The in-guest half (kiosk `cage` + Xwayland on a virtio-gpu scanout) is shared with WHP; `chefer-vz-helper --gui` shows the VM in an AppKit window via `VZVirtualMachineView` (1280×800 default, `CHEFER_VZ_GUI_SIZE=WxH` override) with USB keyboard/absolute-pointer devices. Real-hardware results: the compositor came up **8/8 GUI boots** (the `WLR_LIBINPUT_NO_DEVICES` boot-race fix holds on VZ); X11 clients connect and render; the host↔guest **clipboard** works (text verified both ways; PNG guest→host byte-exact on `NSPasteboard`, host→guest wire-confirmed). Resolution: `automaticallyReconfiguresDisplay` (macOS 14+) does push the window's *pixel* size to the guest — the **initial** mode matches the window's Retina backing exactly (a 1280×800-point window yields a 2560×1600 guest mode; `CHEFER_VZ_GUI_SIZE=1024x600` yields 2048×1200) — and live resize now **re-modesets the guest**: `cage` doesn't follow mode changes by itself, so the guest agent's resize watcher listens for the drm hotplug, re-probes the new preferred mode, and applies it over wlr-output-management (needs the cage ≥ 0.2.0 overlay from the current `build-gui-overlay.sh`; verified end-to-end on real hardware with `CHEFER_VZ_DYNAMIC_RESOLUTION=1`: `xdpyinfo` followed 2560×1600 → 2200×1400). The **default** stays view-scaling — Retina-pixel modes halve Linux UI sizes until the compositor gains HiDPI output scale; the opt-in flips it to true dynamic resolution. Interactive checks that still need a human hand (`scripts/vz-smoke.sh --gui` checklist): typing/clicking into the guest, close-window-ends-app. Known caveat: the clipboard channel (helper → `<vm_ip>:55381` over VZ NAT) sometimes sits in a silent `.waiting` state for up to a minute or never connects — suspected macOS Local Network privacy gating of the ad-hoc-signed helper; needs a stable signing identity and a visible hint in the helper. Design in [docs/DESIGN.md](docs/DESIGN.md).
- [x] **GUI apps on macOS** — *validated on real Apple Silicon (2026-07).* The in-guest half (kiosk `cage` + Xwayland on a virtio-gpu scanout) is shared with WHP; `chefer-vz-helper --gui` shows the VM in an AppKit window via `VZVirtualMachineView` (1280×800 default, `CHEFER_VZ_GUI_SIZE=WxH` override) with USB keyboard/absolute-pointer devices. Real-hardware results: the compositor came up **8/8 GUI boots** (the `WLR_LIBINPUT_NO_DEVICES` boot-race fix holds on VZ); X11 clients connect and render; the host↔guest **clipboard** works (text verified both ways; PNG guest→host byte-exact on `NSPasteboard`, host→guest wire-confirmed). Resolution: `automaticallyReconfiguresDisplay` (macOS 14+) does push the window's *pixel* size to the guest — the **initial** mode matches the window's Retina backing exactly (a 1280×800-point window yields a 2560×1600 guest mode; `CHEFER_VZ_GUI_SIZE=1024x600` yields 2048×1200) — and live resize now **re-modesets the guest**: `cage` doesn't follow mode changes by itself, so the guest agent's resize watcher listens for the drm hotplug, re-probes the new preferred mode, and applies it over wlr-output-management (needs the cage ≥ 0.2.0 overlay from the current `build-gui-overlay.sh`; verified end-to-end on real hardware with `CHEFER_VZ_DYNAMIC_RESOLUTION=1`: `xdpyinfo` followed 2560×1600 → 2200×1400). True dynamic resolution is now the **default** on macOS 14+: the guest re-modesets on window resize and applies the host's HiDPI output scale (the helper pushes the screen's `backingScaleFactor` via kernel cmdline `chefer.gui_scale`; the resize watcher sets it at startup and on every re-modeset), so Retina-pixel modes no longer halve Linux UI sizes — verified end-to-end on real hardware (boot `output scale -> 2`; `xdpyinfo` logical 1280×800 → 1100×700 across a live resize, never the physical pixel sizes; hand-checks passed: cursor/UI size, pointer mapping, upscale quality). `CHEFER_VZ_DYNAMIC_RESOLUTION=0` (or macOS 13, or an old cage <0.2.0 overlay where the guest can't re-modeset) falls back to view-scaling with the aspect lock. Interactive checks that still need a human hand (`scripts/vz-smoke.sh --gui` checklist): typing/clicking into the guest, close-window-ends-app. Known caveat: the clipboard channel (helper → `<vm_ip>:55381` over VZ NAT) sometimes sits in a silent `.waiting` state for up to a minute or never connects — suspected macOS Local Network privacy gating of the ad-hoc-signed helper; needs a stable signing identity and a visible hint in the helper. Design in [docs/DESIGN.md](docs/DESIGN.md).
- [x] **GPU access / CUDA compute** — *opt-in `gpu: true`; validated end-to-end on both WSL2 and native NVIDIA Linux.* Set `gpu: true` on a service and Chefer binds the host's render/compute device nodes into that service's container — on **native Linux** `/dev/nvidia*` + `/dev/dri`; on the **Windows WSL2 backend** `/dev/dxg` + a bind of the host `/usr/lib/wsl/lib` **and `/usr/lib/wsl/drivers`** (its host-version-matched `libdxcore`/`libcuda`/… ) → CUDA/DirectML/OpenCL/Vulkan work with nothing extra in the image. The same passthrough also covers **NVENC/NVDEC video** and OpenCL/ROCm compute; **GL/Vulkan *rendering* does not work in a headless container** (see the native note below). **Validated end-to-end on real WSL2** (GeForce GT 1030) — the container sees `/dev/dxg` + `/usr/lib/wsl/lib` + `/usr/lib/wsl/drivers`, runs `nvidia-smi` with live telemetry, and executes a `nvcc`-built CUDA `vectorAdd` on the GPU (`sm_61`, 1048576 elems, 0 mismatches); binding `/usr/lib/wsl/lib` alone is not enough — its `libcuda`/`libnvidia-ml` are loader shims that dlopen the real driver from `/usr/lib/wsl/drivers`, so both must be bound (else `nvidia-smi` reports "Driver Not Loaded"). `/dev/dri` is bound when present (absent on some WSL kernels; CUDA uses `/dev/dxg`). It is a **clear startup error** on the non-WSL Windows (WHP) micro-VM and the macOS VM — a bare WHP/VZ VM can't tap Microsoft/Apple GPU paravirtualization, so GPU compute stays **native-Linux + WSL2 only**. **Native Linux** — host-driver-version-matched userspace lib injection (à la the NVIDIA Container Toolkit) reads the driver version from `/proc/driver/nvidia/version` and binds the matching NVIDIA driver libs (`libcuda`/`libnvidia-*`/`libnvcuvid`/`libnvoptix`/GL, matching `nvidia-container-cli list`) into `/run/chefer-nvidia` (versioned + real DT_SONAME + `.so`) plus `LD_LIBRARY_PATH`. **Validated on a GeForce RTX 4070** (driver 570.211.01): a `nvcc`-built CUDA vectorAdd from a stock `nvidia/cuda` image that ships no libcuda ran on the GPU (`sm_89`, 1048576 elems, 0 mismatches), and `ffmpeg -c:v h264_nvenc` encoded in-container (NVENC, exit 0). **OpenGL/Vulkan *rendering* does not work in a headless container**, though: NVIDIA's Vulkan ICD (`libGLX_nvidia` → `libnvidia-glcore`) needs a real graphics/X environment (it references Xorg's `ErrorF` and glibc-2.34-removed `__malloc_hook`), so it falls back to `llvmpipe` — and `docker --gpus all` with `NVIDIA_DRIVER_CAPABILITIES=all` (the nvidia-container-toolkit reference) fails identically on the same headless host, so this is a host/driver limitation, not a Chefer one (Chefer's GUI path renders via X11/Wayland socket passthrough + software rasterization instead). No path to Apple NPU / CoreML from a Linux guest.
Loading
Loading