diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 5a910129..fcb91c13 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -40,8 +40,25 @@ jobs: with: upload: true + # Wasm is architecture-independent, so one leg builds the extension modules + # every platform's release points at. + build-extensions: + needs: [verify-tag] + runs-on: blacksmith-8vcpu-ubuntu-2404 + steps: + - uses: actions/checkout@v7 + - uses: DeterminateSystems/nix-installer-action@main + - uses: DeterminateSystems/magic-nix-cache-action@main + - run: ./bin/extensions + - uses: actions/upload-artifact@v7 + with: + name: extensions + path: | + extensions/dist/*.wasm + extensions/dist/manifest.json + release: - needs: [build-packages, build-windows] + needs: [build-packages, build-windows, build-extensions] runs-on: blacksmith-4vcpu-ubuntu-2404 steps: - uses: actions/download-artifact@v8 @@ -61,12 +78,21 @@ jobs: name: windows-x86_64 path: windows + - uses: actions/download-artifact@v8 + with: + name: extensions + path: extensions + - name: List artifacts run: | ls -lh debs/ ls -lh tarballs/ ls -lh windows/ + ls -lh extensions/ + # Release assets are the durable home for a module: Pages only ever + # serves the current release, so a `#digest` pin outlives its version + # here and nowhere else. - name: Create GitHub Release uses: softprops/action-gh-release@v3 with: @@ -74,6 +100,8 @@ jobs: debs/*.deb tarballs/*.tar.gz windows/*.zip + extensions/*.wasm + extensions/manifest.json generate_release_notes: true publish-crates: @@ -144,7 +172,7 @@ jobs: client-payload: '{"version": "${{ github.ref_name }}"}' apt-repo: - needs: [build-packages, build-windows] + needs: [build-packages, build-windows, build-extensions] runs-on: blacksmith-4vcpu-ubuntu-2404 steps: - uses: actions/checkout@v7 @@ -166,6 +194,11 @@ jobs: name: windows-x86_64 path: windows + - uses: actions/download-artifact@v8 + with: + name: extensions + path: extensions + - name: Import GPG key run: echo "$GPG_PRIVATE_KEY" | gpg --batch --import env: @@ -189,6 +222,15 @@ jobs: cp "$f" repo/bin/ done + # Extension modules, for `blit ext run https://install.blit.sh/ext/…`. + # Like repo/bin, this is the current release only: Pages publishes + # the tree wholesale, so an older module's URL stops resolving. The + # manifest carries each module's BLAKE3, which is what a `#digest` + # pin needs and what the GitHub Release keeps forever. + mkdir -p repo/ext + cp extensions/*.wasm repo/ext/ + cp extensions/manifest.json repo/ext/manifest.json + # Write version to /latest echo "${GITHUB_REF_NAME#v}" > repo/latest diff --git a/.gitignore b/.gitignore index c3b7f2cc..4f49c4a5 100644 --- a/.gitignore +++ b/.gitignore @@ -17,3 +17,4 @@ coverage/ dist node_modules/ node_modules +/extensions/target/ diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f90f62b1..a1cd6b1f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -147,21 +147,29 @@ There is no `rustfmt.toml` or `.clippy.toml` — default rustfmt, prettier, and | `gateway` | Runs `blit gateway` with WebSocket + WebTransport (pass=`dev`) | `127.0.0.1:10001` | | `ui` | Vite dev server for `js/ui/` | `127.0.0.1:10000` | | `website` | Astro dev server for `js/website/` | `127.0.0.1:10002` | +| `extensions` | Builds `extensions/dist`, then serves it as a CORS extension registry | `127.0.0.1:10003` | ### Running multiple dev stacks -Every port and socket path is derived from `DEV_INSTANCE` (default `0`). Each instance gets a block of ports at `10000 + (N * 5)`: +Every port and socket path is derived from `DEV_INSTANCE` (default `0`). Each instance gets a block of ports at `10000 + (N * 10)`: -| Instance | UI | Gateway | Website | -| -------- | ----- | ------- | ------- | -| 0 | 10000 | 10001 | 10002 | -| 1 | 10005 | 10006 | 10007 | -| 2 | 10010 | 10011 | 10012 | +| Instance | UI | Gateway | Website | Extensions | +| -------- | ----- | ------- | ------- | ---------- | +| 0 | 10000 | 10001 | 10002 | 10003 | +| 1 | 10010 | 10011 | 10012 | 10013 | +| 2 | 10020 | 10021 | 10022 | 10023 | ```bash -DEV_INSTANCE=1 ./bin/dev # second stack on 10005-10007 +DEV_INSTANCE=1 ./bin/dev # second stack on 10010-10013 ``` +The UI dev server proxies `/ext` to its own instance's registry, so the +Extensions tab of a remote offers the modules _this_ stack built rather than +the published ones. It goes through the page's origin rather than straight to +`127.0.0.1:10003` so that it also works when the dev UI is reached through a +tunnel or reverse proxy, where there is no port to derive from and the registry +port is not published. + `bin/dev` prints the concrete addresses on startup. `DEV_INSTANCE` is intentionally unprefixed: blit strips most `BLIT_*` variables from child terminals, but passes everything else through. This means `DEV_INSTANCE` propagates into nested shells so you always know which instance you're inside and can pick a different one. You can also override individual values: | Variable | Default (instance 0) | Description | @@ -172,6 +180,7 @@ DEV_INSTANCE=1 ./bin/dev # second stack on 10005-10007 | `BLIT_DEV_GW_PORT` | `10001` | Gateway TCP + WebTransport UDP port | | `BLIT_DEV_WT_ADDR` | `:10001` | Browser-facing WebTransport `hostname:port` or `:port` | | `BLIT_DEV_SITE_PORT` | `10002` | Astro website dev-server port | +| `BLIT_DEV_EXT_PORT` | `10003` | Extension registry (`extensions/dist`) port | ## Project structure diff --git a/Cargo.toml b/Cargo.toml index 3d351e44..c553c1af 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -24,7 +24,7 @@ members = [ # A path dependency living under the workspace root would otherwise become an # implicit member, so `--workspace` clippy/test/fmt runs would lint, test, and # reformat third-party code we only consume. -exclude = ["vendor/cros-codecs", "vendor/wayland-backend"] +exclude = ["vendor/cros-codecs", "vendor/wayland-backend", "extensions"] [workspace.package] edition = "2024" diff --git a/README.md b/README.md index 7e62af65..1d8f0717 100644 --- a/README.md +++ b/README.md @@ -76,6 +76,8 @@ Control terminals programmatically: blit terminal start htop # start a terminal, print its ID blit terminal show 1 # dump current terminal text blit terminal send 1 q # send keystrokes +blit terminal journal 1 # commands the shell has run (needs OSC 133) +blit terminal output 1 --wait 60 # that command's output ``` Inspect and disconnect other clients attached to the same server: diff --git a/SKILL.md b/SKILL.md index 6a59bd35..ebd9daa3 100644 --- a/SKILL.md +++ b/SKILL.md @@ -8,7 +8,8 @@ description: > language server for definitions, references, hover, completions, signature help, symbols, diagnostics, or a rename plan; read or write the server's key/value store; or run and interact with GUI applications. Covers - starting PTYs, sending keystrokes, reading output, checking exit status, + starting PTYs, sending keystrokes, reading output, listing the commands a + shell has run, waiting on a command's output, checking exit status, managing terminal lifecycle, attaching a local terminal to a remote one, searching file contents across a tree, and driving graphical windows through the experimental headless Wayland compositor (listing surfaces, capturing screenshots, clicking, typing, @@ -32,6 +33,10 @@ filesystem, git, and language-server commands. Beyond terminals it can: backed by real language servers running next to the code. - **Key/value store** — `blit kv get|put|rm|ls`, prefix-watchable, with compare-and-swap writes. Handy as host-local scratch space for scripts. +- **Command journal** — `blit terminal journal|output` and + `history --since`, given a shell that emits OSC 133 (see + `docs/shell-integration.md`). `wait --pattern` matches only output + produced after the wait began. ## Install diff --git a/bin/dev b/bin/dev index 75fe7481..763dcb01 100755 --- a/bin/dev +++ b/bin/dev @@ -8,9 +8,9 @@ cd "$(dirname "$0")/.." # Set DEV_INSTANCE to run multiple dev stacks side-by-side. Each instance # gets a block of ports starting at 10000 + (N * 10): # -# Instance 0 (default): 10000 10001 10002 -# Instance 1: 10010 10011 10012 -# Instance 2: 10020 10021 10022 +# Instance 0 (default): 10000 10001 10002 10003 +# Instance 1: 10010 10011 10012 10013 +# Instance 2: 10020 10021 10022 10023 # ... # # The blit server Unix socket is /tmp/blit-dev.sock for instance 0, @@ -32,9 +32,11 @@ N="${DEV_INSTANCE:-0}" if [ "$N" -eq 0 ]; then export BLIT_DEV_SOCK="${BLIT_DEV_SOCK:-/tmp/blit-dev.sock}" + export BLIT_DEV_EXT_DB="${BLIT_DEV_EXT_DB:-/tmp/blit-dev-ext/extensions.redb}" PC_SOCK="/tmp/blit-dev-pc.sock" else export BLIT_DEV_SOCK="${BLIT_DEV_SOCK:-/tmp/blit-dev-${N}.sock}" + export BLIT_DEV_EXT_DB="${BLIT_DEV_EXT_DB:-/tmp/blit-dev-${N}-ext/extensions.redb}" PC_SOCK="/tmp/blit-dev-pc-${N}.sock" fi @@ -42,6 +44,12 @@ BASE=$(( 10000 + N * 10 )) export BLIT_DEV_UI_PORT="${BLIT_DEV_UI_PORT:-$(( BASE ))}" export BLIT_DEV_GW_PORT="${BLIT_DEV_GW_PORT:-$(( BASE + 1 ))}" export BLIT_DEV_SITE_PORT="${BLIT_DEV_SITE_PORT:-$(( BASE + 2 ))}" +# The stack's own extension registry: extensions/dist served with CORS, which +# is what the UI's Extensions panel installs from in dev. The vite dev server +# is told this port and proxies /ext to it, so a second instance's page reaches +# that instance's modules and not the first one's — and a page reached through +# a tunnel reaches them at all, which a bare loopback port cannot offer. +export BLIT_DEV_EXT_PORT="${BLIT_DEV_EXT_PORT:-$(( BASE + 3 ))}" # Browser-facing WebTransport authority. A leading `:port` keeps the browser # page hostname; `hostname:port` overrides both. export BLIT_DEV_WT_ADDR="${BLIT_DEV_WT_ADDR:-:${BLIT_DEV_GW_PORT}}" diff --git a/bin/ext-registry b/bin/ext-registry new file mode 100755 index 00000000..549f9044 --- /dev/null +++ b/bin/ext-registry @@ -0,0 +1,65 @@ +#!/usr/bin/env node +/** + * Serve `extensions/dist` as an extension registry, the way + * https://install.blit.sh/ext serves one: `manifest.json` plus the modules it + * names, with CORS open so a page on the vite dev port can install from it. + * + * Node rather than a static-file package because the dev shell has node and + * nothing else guaranteed — no python3 — and this is thirty lines. + * + * bin/ext-registry [port] [directory] + */ + +import { createReadStream, statSync } from "node:fs"; +import { createServer } from "node:http"; +import { extname, join, normalize, resolve } from "node:path"; + +const port = Number(process.argv[2] ?? process.env.BLIT_DEV_EXT_PORT ?? 10003); +const root = resolve(process.argv[3] ?? "extensions/dist"); + +const TYPES = { + ".wasm": "application/wasm", + ".json": "application/json", + ".br": "application/octet-stream", +}; + +createServer((request, response) => { + const cors = { + "access-control-allow-origin": "*", + "access-control-allow-headers": "*", + }; + if (request.method === "OPTIONS") { + response.writeHead(204, cors).end(); + return; + } + // Everything under the root, and nothing above it: a leading `..` in the + // request path must not be able to name the rest of the checkout. + const requested = decodeURIComponent( + new URL(request.url ?? "/", "http://localhost").pathname, + ); + const path = join(root, normalize(requested).replace(/^(\.\.[/\\])+/, "")); + if (!path.startsWith(root)) { + response.writeHead(403, cors).end("forbidden\n"); + return; + } + let size; + try { + const stat = statSync(path); + if (!stat.isFile()) throw new Error("not a file"); + size = stat.size; + } catch { + response.writeHead(404, cors).end("not found\n"); + return; + } + response.writeHead(200, { + ...cors, + "content-type": TYPES[extname(path)] ?? "application/octet-stream", + "content-length": String(size), + // The digest is the identity, so a stale copy is a wrong install rather + // than an old one. In a dev stack the modules are rebuilt constantly. + "cache-control": "no-store", + }); + createReadStream(path).pipe(response); +}).listen(port, () => { + console.log(`extension registry: http://127.0.0.1:${port} -> ${root}`); +}); diff --git a/bin/extensions b/bin/extensions new file mode 100755 index 00000000..e3316ad1 --- /dev/null +++ b/bin/extensions @@ -0,0 +1,4 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")/.." +exec nix run .#extensions -- "$@" diff --git a/crates/alacritty-driver/src/lib.rs b/crates/alacritty-driver/src/lib.rs index 75651ab4..38a42f1a 100644 --- a/crates/alacritty-driver/src/lib.rs +++ b/crates/alacritty-driver/src/lib.rs @@ -416,6 +416,26 @@ struct SearchCandidate { // scrollback — which is also the first moment a client can be parked in it. const SCROLL_PROBE: usize = 1; +/// A sequence-addressed read of the grid, and the cursor to resume from. +/// +/// `next_seq`/`next_col` fed back into the next [`TerminalDriver::seq_text`] +/// return exactly what was appended in between — including the rest of a +/// line that was still being written when the first read happened. +#[derive(Clone, Debug, Default, PartialEq, Eq)] +pub struct SeqText { + pub text: String, + /// Where the returned text actually starts, after clamping to what the + /// scrollback still holds. + pub start_seq: u64, + pub start_col: u16, + pub next_seq: u64, + pub next_col: u16, + /// `max_bytes` cut the read short; `next_seq` names the first row left out. + pub truncated: bool, + /// The requested start had already been evicted from the scrollback. + pub evicted: bool, +} + pub struct TerminalDriver { term: Term, processor: alacritty_terminal::vte::ansi::Processor, @@ -429,6 +449,17 @@ pub struct TerminalDriver { /// Lines rotated out of the viewport since this terminal was created. /// Monotonic; consumers track their own last-seen value and use deltas. scrolled_lines: u64, + /// The same motion counted absolutely, for content identity rather than + /// for re-anchoring a view. + /// + /// `scrolled_lines` cannot serve: its probe can only be armed once a line + /// has reached the scrollback, so however many lines rotate out before + /// the scrollback exists go uncounted. That is harmless for a delta, and + /// fatal for an absolute address — a line would change its number the + /// first time the terminal scrolled. This counter closes the gap with the + /// growth of the history, which measures exactly the motion the probe + /// cannot see. + rotated_lines: u64, } impl TerminalDriver { @@ -455,14 +486,16 @@ impl TerminalDriver { used_rows: 0, used_rows_dirty: true, scrolled_lines: 0, + rotated_lines: 0, } } pub fn process(&mut self, data: &[u8]) { let used_rows_action = self.modes.process(data); + let history_before = self.history_len(); self.arm_scroll_probe(); self.processor.advance(&mut self.term, data); - self.read_scroll_probe(); + self.read_scroll_probe(history_before); if used_rows_action == UsedRowsAction::Reset { self.reset_used_rows(); } @@ -489,9 +522,20 @@ impl TerminalDriver { self.term.grid_mut().scroll_display(Scroll::Delta(delta)); } - fn read_scroll_probe(&mut self) { + fn history_len(&self) -> usize { + let grid = self.term.grid(); + grid.total_lines().saturating_sub(grid.screen_lines()) + } + + fn read_scroll_probe(&mut self, history_before: usize) { let after = self.term.grid().display_offset(); - self.scrolled_lines += after.saturating_sub(SCROLL_PROBE) as u64; + let probed = after.saturating_sub(SCROLL_PROBE) as u64; + self.scrolled_lines += probed; + // Whichever of the two saw more motion. Before the scrollback + // exists only the growth sees any; once it is full only the probe + // does; in between they agree. + let grown = self.history_len().saturating_sub(history_before) as u64; + self.rotated_lines += probed.max(grown); } pub fn size(&self) -> (u16, u16) { @@ -504,12 +548,28 @@ impl TerminalDriver { cols: cols as usize, rows: rows as usize, }; + let history_before = self.history_len() as i64; self.term.resize(dims); // A resize shifts the history around on its own (rewrap, and lines // pushed out when the viewport shrinks). That motion isn't the app // scrolling and the client re-derives its geometry from the next // frame anyway, so re-arm the probe without counting it. self.arm_scroll_probe(); + // Sequences do have to follow it, though. A shorter viewport pushes + // rows into the history (`history_len` grows); a taller one pulls + // them back out (`grow_lines` does `cursor.line += from_history` and + // `history_len` shrinks). `saturating_sub` would miss the grow + // direction, so `cursor_seq` and every already-captured record would + // jump by `from_history` rows. The signed delta cancels the cursor + // move: shrink increments `rotated_lines`, grow decrements it. + // A column change rewraps and no such correspondence survives — + // see docs/design/term-journal.md § Resize. + let delta = self.history_len() as i64 - history_before; + if delta >= 0 { + self.rotated_lines += delta as u64; + } else { + self.rotated_lines = self.rotated_lines.saturating_sub((-delta) as u64); + } let capped = self.used_rows.min(rows); if capped != self.used_rows { self.used_rows = capped; @@ -687,6 +747,34 @@ impl TerminalDriver { frame } + /// One grid row rendered to text, with whether it soft-wraps into the + /// next. `c0`/`c1` are inclusive column bounds, already clamped. + /// + /// A soft-wrapped row keeps its trailing space: it's the gap between + /// words ("for all", not "forall"). + fn row_text(&self, line: Line, c0: usize, c1: usize) -> (String, bool) { + let grid_row = &self.term.grid()[line]; + let mut text = String::new(); + for col_idx in c0..=c1 { + let cell = &grid_row[Column(col_idx)]; + if cell.flags.contains(CellFlags::WIDE_CHAR_SPACER) { + continue; + } + let c = if cell.c == '\0' { ' ' } else { cell.c }; + text.push(c); + for &zw in cell.zerowidth().into_iter().flatten() { + text.push(zw); + } + } + let wrapped = grid_row + .last() + .is_some_and(|c| c.flags.contains(CellFlags::WRAPLINE)); + if !wrapped { + text.truncate(text.trim_end().len()); + } + (text, wrapped) + } + pub fn get_text_range( &self, start_tail: u32, @@ -713,8 +801,6 @@ impl TerminalDriver { let end_i = end_line.0.min(last_line.0); while line_i <= end_i { - let line_idx = Line(line_i); - let grid_row = &grid[line_idx]; let c0 = if line_i == start_line.0 { (start_col as usize).min(cols.saturating_sub(1)) } else { @@ -726,27 +812,8 @@ impl TerminalDriver { cols.saturating_sub(1) }; - let mut line_text = String::new(); - for col_idx in c0..=c1 { - let cell = &grid_row[Column(col_idx)]; - if cell.flags.contains(CellFlags::WIDE_CHAR_SPACER) { - continue; - } - let c = if cell.c == '\0' { ' ' } else { cell.c }; - line_text.push(c); - for &zw in cell.zerowidth().into_iter().flatten() { - line_text.push(zw); - } - } - let is_wrapped = grid_row - .last() - .is_some_and(|c| c.flags.contains(CellFlags::WRAPLINE)); - // Keep a soft-wrapped row's trailing space: it's the gap between words ("for all", not "forall"). - if is_wrapped { - result.push_str(&line_text); - } else { - result.push_str(line_text.trim_end()); - } + let (line_text, is_wrapped) = self.row_text(Line(line_i), c0, c1); + result.push_str(&line_text); if line_i < end_i && !is_wrapped { result.push('\n'); } @@ -756,6 +823,112 @@ impl TerminalDriver { result } + /// Absolute sequence and column of the cursor: where the next byte the + /// application writes will land. + /// + /// A sequence is `scrolled_lines + row`, so it names the same text for as + /// long as that text is retained — the property grid coordinates lack, + /// since every scroll renumbers them. + pub fn cursor_seq(&self) -> (u64, u16) { + let cursor = self.term.grid().cursor.point; + let row = cursor.line.0.max(0) as u64; + (self.rotated_lines + row, cursor.column.0 as u16) + } + + /// The oldest sequence still in the scrollback. Anything below it has + /// been evicted and can no longer be read. + pub fn oldest_seq(&self) -> u64 { + self.rotated_lines.saturating_sub(self.history_len() as u64) + } + + /// Text from `(from_seq, from_col)` up to `end_seq` exclusive, or up to + /// and including the cursor's line when `end_seq` is `None`. + /// + /// `max_bytes` is a soft cap: a row is never split, so the result can + /// overshoot by at most one row. That keeps paging monotonic — a client + /// re-asking from `next_seq` always makes progress and never skips a + /// line, which a hard byte cut could not promise. + pub fn seq_text( + &self, + from_seq: u64, + from_col: u16, + end_seq: Option, + max_bytes: usize, + ) -> SeqText { + let grid = self.term.grid(); + let screen = grid.screen_lines(); + let cols = grid.columns(); + let (cursor_seq, cursor_col) = self.cursor_seq(); + let oldest = self.oldest_seq(); + + let mut out = SeqText { + start_seq: from_seq, + start_col: from_col, + next_seq: cursor_seq, + next_col: cursor_col, + ..SeqText::default() + }; + if screen == 0 || cols == 0 { + return out; + } + + // The bottom of the grid bounds every read; beyond it there is no + // text yet, only cells nothing has written to. + let last_seq = self.rotated_lines + screen as u64 - 1; + let end_inclusive = match end_seq { + Some(end) => end.saturating_sub(1).min(last_seq), + None => cursor_seq.min(last_seq), + }; + // A bounded range that is already history answers from history; only + // an open-ended read resumes at the live cursor. + let (done_seq, done_col) = match end_seq { + Some(end) => (end.min(last_seq + 1), 0), + None => (cursor_seq, cursor_col), + }; + out.next_seq = done_seq; + out.next_col = done_col; + + let mut seq = from_seq; + if seq < oldest { + seq = oldest; + out.start_col = 0; + out.evicted = true; + } + out.start_seq = seq; + if seq > end_inclusive { + return out; + } + + let mut prev_wrapped = false; + let mut first = true; + while seq <= end_inclusive { + let line = Line((seq as i64 - self.rotated_lines as i64) as i32); + let c0 = if seq == out.start_seq { + (out.start_col as usize).min(cols - 1) + } else { + 0 + }; + let (row, wrapped) = self.row_text(line, c0, cols - 1); + let sep = usize::from(!first && !prev_wrapped); + // The first row always goes in, budget or not, so that a client + // paging through output cannot stall on an over-long line. + if !first && out.text.len() + sep + row.len() > max_bytes { + out.truncated = true; + out.next_seq = seq; + out.next_col = 0; + return out; + } + if sep == 1 { + out.text.push('\n'); + } + out.text.push_str(&row); + prev_wrapped = wrapped; + first = false; + seq += 1; + } + out + } + pub fn search_result(&self, query: &str) -> Option { let query = query.trim(); if query.is_empty() { diff --git a/crates/alacritty-driver/tests/seq_cursor.rs b/crates/alacritty-driver/tests/seq_cursor.rs new file mode 100644 index 00000000..4a48bc7e --- /dev/null +++ b/crates/alacritty-driver/tests/seq_cursor.rs @@ -0,0 +1,262 @@ +//! A client that polls a terminal wants "what is new since I last looked", +//! not the whole grid. These tests pin down the sequence cursor that answers +//! it: that it never loses a line, never repeats one, survives eviction +//! visibly, and resumes mid-line when the application stopped mid-line. + +use blit_alacritty::TerminalDriver; + +fn feed_lines(d: &mut TerminalDriver, range: std::ops::Range) { + for i in range { + d.process(format!("line {i}\r\n").as_bytes()); + } +} + +#[test] +fn cursor_returns_only_what_was_appended() { + let mut d = TerminalDriver::new(10, 40, 1000); + feed_lines(&mut d, 0..5); + + let (seq, col) = d.cursor_seq(); + feed_lines(&mut d, 5..8); + + // The trailing newline is the empty row the cursor now sits on. Keeping + // it is what lets successive reads be concatenated without inventing a + // separator that may or may not belong. + let read = d.seq_text(seq, col, None, 64 * 1024); + assert_eq!(read.text, "line 5\nline 6\nline 7\n"); + assert!(!read.truncated); + assert!(!read.evicted); + + // Reading again from the returned cursor yields nothing new. + let again = d.seq_text(read.next_seq, read.next_col, None, 64 * 1024); + assert_eq!(again.text, ""); + assert_eq!( + (again.next_seq, again.next_col), + (read.next_seq, read.next_col) + ); +} + +#[test] +fn cursor_resumes_inside_a_partial_line() { + let mut d = TerminalDriver::new(10, 40, 1000); + d.process(b"Continue? "); + + let (seq, col) = d.cursor_seq(); + let first = d.seq_text(0, 0, None, 64 * 1024); + assert_eq!(first.text, "Continue?"); + assert_eq!((first.next_seq, first.next_col), (seq, col)); + + d.process(b"[y/N] "); + let rest = d.seq_text(first.next_seq, first.next_col, None, 64 * 1024); + assert_eq!( + rest.text, "[y/N]", + "the resumed read must not repeat the prompt it already returned" + ); +} + +#[test] +fn a_prompt_written_without_a_newline_is_visible_immediately() { + // The reason the cursor line is included rather than withheld until it + // is complete: an agent waiting on "[y/N]" would otherwise wait forever. + let mut d = TerminalDriver::new(10, 40, 1000); + feed_lines(&mut d, 0..3); + let (seq, col) = d.cursor_seq(); + + d.process(b"overwrite? [y/N] "); + let read = d.seq_text(seq, col, None, 64 * 1024); + assert_eq!(read.text, "overwrite? [y/N]"); +} + +#[test] +fn a_bounded_range_reads_one_commands_output() { + let mut d = TerminalDriver::new(10, 40, 1000); + feed_lines(&mut d, 0..3); + let start = d.cursor_seq().0; + feed_lines(&mut d, 3..6); + let end = d.cursor_seq().0; + feed_lines(&mut d, 6..9); + + let read = d.seq_text(start, 0, Some(end), 64 * 1024); + assert_eq!(read.text, "line 3\nline 4\nline 5"); + assert_eq!(read.next_seq, end, "a bounded read stops where it was told"); + assert_eq!(read.next_col, 0); +} + +#[test] +fn eviction_is_reported_not_silently_misread() { + let mut d = TerminalDriver::new(10, 40, 20); + feed_lines(&mut d, 0..200); + + let oldest = d.oldest_seq(); + assert!(oldest > 0, "a 20-line scrollback must have evicted by now"); + + let read = d.seq_text(0, 0, None, 64 * 1024); + assert!(read.evicted, "reading from an evicted start must say so"); + assert_eq!(read.start_seq, oldest); + assert!( + !read.text.contains("line 0\n"), + "evicted text cannot be conjured back" + ); + + let fresh = d.seq_text(oldest, 0, None, 64 * 1024); + assert!(!fresh.evicted); +} + +#[test] +fn truncation_pages_forward_without_losing_a_line() { + let mut d = TerminalDriver::new(10, 40, 1000); + feed_lines(&mut d, 0..40); + + let mut cursor = (0u64, 0u16); + let mut collected = String::new(); + let mut rounds = 0; + loop { + // 20 bytes holds two "line N" rows, so this pages many times. + let read = d.seq_text(cursor.0, cursor.1, None, 20); + if !read.text.is_empty() { + if !collected.is_empty() { + collected.push('\n'); + } + collected.push_str(&read.text); + } + let next = (read.next_seq, read.next_col); + if !read.truncated { + break; + } + assert_ne!(next, cursor, "a truncated page must make progress"); + cursor = next; + rounds += 1; + assert!(rounds < 200, "paging failed to terminate"); + } + + for i in 0..40 { + assert!( + collected.contains(&format!("line {i}")), + "paged reads dropped line {i}" + ); + } + assert!(rounds > 1, "the budget should have forced several pages"); +} + +#[test] +fn a_cursor_from_the_future_yields_nothing() { + let mut d = TerminalDriver::new(10, 40, 1000); + feed_lines(&mut d, 0..3); + let read = d.seq_text(u64::MAX, 0, None, 64 * 1024); + assert_eq!(read.text, ""); + assert_eq!(read.next_seq, d.cursor_seq().0); +} + +#[test] +fn blank_lines_survive_the_round_trip() { + let mut d = TerminalDriver::new(10, 40, 1000); + let (seq, col) = d.cursor_seq(); + d.process(b"a\r\n\r\n\r\nb\r\n"); + let read = d.seq_text(seq, col, None, 64 * 1024); + assert_eq!(read.text, "a\n\n\nb\n", "empty rows are still rows"); +} + +#[test] +fn soft_wrapped_rows_stay_one_logical_line() { + let mut d = TerminalDriver::new(10, 10, 1000); + let (seq, col) = d.cursor_seq(); + d.process(b"aaaaaaaaaabbbbb\r\n"); + let read = d.seq_text(seq, col, None, 64 * 1024); + assert_eq!(read.text, "aaaaaaaaaabbbbb\n"); +} + +#[test] +fn the_first_line_keeps_its_sequence_across_the_first_scroll() { + // The trap this counter exists for: the scroll that creates the + // scrollback is invisible to a probe that needs a scrollback to arm, so + // a naive counter renumbers every retained line exactly once. + let mut d = TerminalDriver::new(4, 40, 1000); + feed_lines(&mut d, 0..3); + let before = d.seq_text(0, 0, Some(1), 64 * 1024); + assert_eq!(before.text, "line 0"); + + feed_lines(&mut d, 3..30); + + let after = d.seq_text(0, 0, Some(1), 64 * 1024); + assert_eq!( + after.text, "line 0", + "sequence 0 must still name the first line after the grid scrolled" + ); + assert_eq!( + d.oldest_seq(), + 0, + "nothing was evicted from a 1000-line scrollback" + ); +} + +#[test] +fn every_line_is_addressable_by_its_own_sequence() { + let mut d = TerminalDriver::new(5, 40, 1000); + feed_lines(&mut d, 0..25); + for i in 0..25u64 { + let read = d.seq_text(i, 0, Some(i + 1), 64 * 1024); + assert_eq!( + read.text, + format!("line {i}"), + "sequence {i} names line {i}" + ); + } +} + +fn named_lines(d: &TerminalDriver, n: u64) -> Vec { + (0..n) + .map(|i| d.seq_text(i, 0, Some(i + 1), 64 * 1024).text) + .collect() +} + +#[test] +fn a_taller_resize_keeps_sequence_identity() { + // Viewport 4, 20 lines of output → 16 lines of history. Growing to 10 + // pulls 6 of those back into the viewport (`grow_lines`); every sequence + // that already named a line must still name it. + let mut d = TerminalDriver::new(4, 40, 1000); + feed_lines(&mut d, 0..20); + let before = named_lines(&d, 20); + let cursor = d.cursor_seq(); + + d.resize(10, 40); + + assert_eq!( + named_lines(&d, 20), + before, + "grow must not renumber retained lines" + ); + assert_eq!( + d.cursor_seq(), + cursor, + "cursor_seq must not jump when history is pulled into the viewport" + ); +} + +#[test] +fn a_shorter_resize_keeps_sequence_identity() { + let mut d = TerminalDriver::new(10, 40, 1000); + feed_lines(&mut d, 0..20); + let before = named_lines(&d, 20); + let cursor = d.cursor_seq(); + + d.resize(4, 40); + + assert_eq!( + named_lines(&d, 20), + before, + "shrink must not renumber retained lines" + ); + assert_eq!(d.cursor_seq(), cursor); +} + +#[test] +fn growing_with_no_history_does_not_invent_rotation() { + let mut d = TerminalDriver::new(10, 40, 1000); + feed_lines(&mut d, 0..3); + let before = named_lines(&d, 3); + let cursor = d.cursor_seq(); + d.resize(20, 40); + assert_eq!(named_lines(&d, 3), before); + assert_eq!(d.cursor_seq(), cursor); +} diff --git a/crates/cli/src/agent.rs b/crates/cli/src/agent.rs index d4e6180b..f0297ef0 100644 --- a/crates/cli/src/agent.rs +++ b/crates/cli/src/agent.rs @@ -2,19 +2,20 @@ use std::collections::HashMap; use blit_remote::{AXIS_SOURCE_FINGER, AXIS_SOURCE_WHEEL, PointerAxisEvent}; use blit_remote::{ - C2S_CLIENT_FEATURES, C2S_SURFACE_ACK, C2S_SURFACE_CAPTURE, C2S_SURFACE_LIST, + C2S_CLIENT_FEATURES, C2S_CLIENT_LIST, C2S_SURFACE_ACK, C2S_SURFACE_CAPTURE, C2S_SURFACE_LIST, C2S_SURFACE_POINTER, CAPTURE_FORMAT_AVIF, CAPTURE_FORMAT_PNG, CODEC_SUPPORT_AV1, CODEC_SUPPORT_AV1_444, CODEC_SUPPORT_H264, CODEC_SUPPORT_H264_444, CREATE2_WANT_STATUS, - EXIT_REASON_NORMAL, EXIT_STATUS_UNKNOWN, FEATURE_CLIENT_CONTROL, FEATURE_CREATE_STATUS, - FEATURE_PTY_DEADLINE, KICK_REASON_MAX, S2C_CLIPBOARD_CONTENT, S2C_CLIPBOARD_LIST, S2C_EXITED, - S2C_HELLO, S2C_KICKED, S2C_LIST, S2C_PING, S2C_QUIT, S2C_READY, S2C_SURFACE_CAPTURE, - S2C_SURFACE_FRAME, S2C_SURFACE_LIST, S2C_TERM_CWD, S2C_TEXT, S2C_TITLE, S2C_UPDATE, STATUS_OK, - SURFACE_FRAME_CODEC_AV1, SURFACE_FRAME_CODEC_MASK, SURFACE_FRAME_FLAG_KEYFRAME, ServerMsg, - TerminalState, exit_reason_text, msg_ack, msg_c2s_clipboard_get, msg_c2s_clipboard_list, - msg_c2s_clipboard_set, msg_c2s_primary_set, msg_client_list, msg_close, msg_create2_full, - msg_deadline, msg_display_rate, msg_input, msg_kick, msg_kill, msg_mouse, msg_quit, msg_read, - msg_resize, msg_restart, msg_subscribe, msg_surface_close, msg_surface_focus, - msg_surface_input, msg_surface_pointer_axis2, msg_surface_resize, msg_surface_subscribe, + EXIT_REASON_NORMAL, EXIT_STATUS_UNKNOWN, FEATURE_CLIENT_CONTROL, FEATURE_CLIENT_ORIGIN, + FEATURE_CREATE_STATUS, FEATURE_PTY_DEADLINE, KICK_REASON_MAX, S2C_CLIPBOARD_CONTENT, + S2C_CLIPBOARD_LIST, S2C_EXITED, S2C_HELLO, S2C_KICKED, S2C_LIST, S2C_PING, S2C_QUIT, S2C_READY, + S2C_SURFACE_CAPTURE, S2C_SURFACE_FRAME, S2C_SURFACE_LIST, S2C_TERM_CWD, S2C_TEXT, S2C_TITLE, + S2C_UPDATE, STATUS_OK, SURFACE_FRAME_CODEC_AV1, SURFACE_FRAME_CODEC_MASK, + SURFACE_FRAME_FLAG_KEYFRAME, ServerMsg, TerminalState, exit_reason_text, msg_ack, + msg_c2s_clipboard_get, msg_c2s_clipboard_list, msg_c2s_clipboard_set, msg_c2s_primary_set, + msg_client_list, msg_client_list_with_origin, msg_close, msg_create2_full, msg_deadline, + msg_display_rate, msg_input, msg_kick, msg_kill, msg_mouse, msg_quit, msg_read, msg_resize, + msg_restart, msg_subscribe, msg_surface_close, msg_surface_focus, msg_surface_input, + msg_surface_pointer_axis2, msg_surface_resize, msg_surface_subscribe, msg_surface_subscribe_ext, msg_surface_subscribe_scaled, msg_surface_text, msg_term_cwd, parse_server_msg, parse_term_cwd_reply, status_text, }; @@ -36,6 +37,9 @@ pub(crate) struct AgentConn { pub(crate) ptys: Vec, pub(crate) titles: HashMap, pub(crate) exited: HashMap, + /// Stamped identity per surface, as `engine:app:instance`, collected from + /// the pre-READY replay. Absent for surfaces on the shared Wayland socket. + pub(crate) surface_origins: HashMap, /// HELLO feature bits — gates requests older servers don't answer. pub(crate) features: u32, /// Reassembly buffer for `S2C_FRAGMENT` messages from the server. @@ -67,6 +71,7 @@ impl AgentConn { let mut ptys = Vec::new(); let mut titles = HashMap::new(); let mut exited = HashMap::new(); + let mut surface_origins = HashMap::new(); let mut features = 0u32; let mut fragment_buf = FragmentReassembly::default(); @@ -95,6 +100,23 @@ impl AgentConn { features = f; } } + // Stamped identity arrives in the pre-READY replay, right after + // each SURFACE_CREATED, so it is here rather than in the + // SURFACE_LIST reply — whose record layout is fixed. + blit_remote::S2C_SURFACE_ORIGIN => { + if let Some(ServerMsg::SurfaceOrigin { + surface_id, + sandbox_engine, + app_id, + instance_id, + }) = parse_server_msg(&data) + { + surface_origins.insert( + surface_id, + format!("{sandbox_engine}:{app_id}:{instance_id}"), + ); + } + } S2C_PING => {} S2C_LIST => { if let Some(ServerMsg::List { entries }) = parse_server_msg(&data) { @@ -135,6 +157,7 @@ impl AgentConn { ptys, titles, exited, + surface_origins, features, fragment_buf, kicked: None, @@ -199,7 +222,10 @@ impl AgentConn { self.ptys.iter().any(|p| p.id == id) } - async fn recv_deadline(&mut self, deadline: tokio::time::Instant) -> Result, String> { + pub(crate) async fn recv_deadline( + &mut self, + deadline: tokio::time::Instant, + ) -> Result, String> { let data = tokio::time::timeout_at( deadline, read_message(&mut self.reader, &mut self.fragment_buf), @@ -275,7 +301,16 @@ pub async fn cmd_clients(transport: Transport) -> Result<(), String> { } const NONCE: u16 = 1; - conn.send(&msg_client_list(NONCE)).await?; + // An older server answers the flags byte with INVALID rather than a + // catalog, so the ORIGIN column is only asked for when it is on offer — + // and prints "unknown" for every row when it is not. + let want_origin = conn.features & FEATURE_CLIENT_ORIGIN != 0; + let request = if want_origin { + msg_client_list_with_origin(C2S_CLIENT_LIST, NONCE) + } else { + msg_client_list(NONCE) + }; + conn.send(&request).await?; loop { let data = conn.recv().await?; // KICK_RESULT is the client-control family's status reply, so a @@ -303,7 +338,9 @@ pub async fn cmd_clients(transport: Transport) -> Result<(), String> { }) = reply && nonce == NONCE { - println!("ID\tAGE_S\tOUT_BYTES_S\tIN_BYTES_S\tSUBSCRIPTIONS\tTERMINALS\tSURFACES"); + println!( + "ID\tAGE_S\tOUT_BYTES_S\tIN_BYTES_S\tSUBSCRIPTIONS\tTERMINALS\tSURFACES\tORIGIN" + ); for client in clients .into_iter() .filter(|client| client.client_id != self_id) @@ -351,8 +388,22 @@ pub async fn cmd_clients(transport: Transport) -> Result<(), String> { }) .collect::>() .join(","); + let origin = match &client.origin { + None => "unknown".to_string(), + Some(blit_remote::ClientOrigin::Network) => "network".to_string(), + Some(blit_remote::ClientOrigin::Extension { + extension_id, name, .. + }) => { + if name.is_empty() { + format!("ext:id:{extension_id:016x}") + } else { + format!("ext:{name}") + } + } + Some(blit_remote::ClientOrigin::Unknown { kind, .. }) => format!("kind:{kind}"), + }; println!( - "{}\t{}\t{}\t{}\t{subscriptions}\t{terminals}\t{surfaces}", + "{}\t{}\t{}\t{}\t{subscriptions}\t{terminals}\t{surfaces}\t{origin}", client.client_id, client.age_secs, client.outbound_bytes_per_sec, @@ -822,75 +873,230 @@ pub(crate) fn exit_code_from_status(status: i32) -> i32 { } } -pub async fn cmd_wait( - transport: Transport, +/// The first line in `pending` that matches, consuming what it has ruled out. +/// +/// The last line is kept whole or partial and re-examined on the next call, +/// so a line still being written can match — the grid scan this replaced +/// saw it too — without a match ever being reported twice. +fn first_match(pending: &mut String, re: ®ex::Regex, max: usize) -> Option { + let tail_at = pending.rfind('\n').map_or(0, |i| i + 1); + for line in pending[..tail_at].lines() { + if re.is_match(line) { + return Some(line.to_string()); + } + } + let tail = pending[tail_at..].to_string(); + if !tail.is_empty() && re.is_match(&tail) { + return Some(tail); + } + *pending = tail; + // A line that never ends must not grow without bound. + if pending.len() > max { + let mut cut = pending.len() - max; + while cut < pending.len() && !pending.is_char_boundary(cut) { + cut += 1; + } + pending.drain(..cut); + } + None +} + +/// Wait for `re` to match output produced *after* this call. +/// +/// The anchor is the server's sequence cursor, taken before subscribing: +/// every read is "what was appended since", so text already on screen cannot +/// satisfy the wait (docs/design/term-journal.md § `wait --pattern`). +async fn wait_for_pattern( + mut conn: AgentConn, id: u16, - timeout_secs: u64, - pattern: Option, + re: ®ex::Regex, + deadline: tokio::time::Instant, ) -> Result { - let mut conn = AgentConn::connect(transport).await?; + use blit_remote::journal::{OUTPUT_TRUNCATED, S2C_TERM_OUTPUT, msg_term_since}; - if !conn.has_pty(id) { - return Err(format!("pty {id} not found")); - } + const PROBE: u16 = 0xF0; + const READ: u16 = 0xF1; + const PENDING_MAX: usize = 64 * 1024; + let max_bytes = crate::journal::OUTPUT_MAX_BYTES; - let deadline = tokio::time::Instant::now() + std::time::Duration::from_secs(timeout_secs); + let (mut seq, mut col) = crate::journal::probe_cursor(&mut conn, id, PROBE).await?; + conn.send(&msg_subscribe(id)).await?; + // Output racing between the probe and the subscription is already behind + // the cursor; reading once now is what stops it being missed if the + // terminal then goes quiet. + conn.send(&msg_term_since(READ, id, seq, col, max_bytes, 0)) + .await?; + let mut in_flight = true; + let mut pending = String::new(); + let mut exited: Option<(i32, u8)> = None; - if let Some(ref pat) = pattern { - let re = regex::Regex::new(pat).map_err(|e| format!("invalid pattern: {e}"))?; + loop { + let data = match conn.recv_deadline(deadline).await { + Ok(d) => d, + Err(e) if e == "timeout" => { + eprintln!("blit: timed out waiting for pty {id}"); + return Ok(124); + } + Err(e) => return Err(e), + }; + if data.is_empty() { + continue; + } + match data[0] { + S2C_UPDATE if data.len() >= 3 => { + if u16::from_le_bytes([data[1], data[2]]) != id { + continue; + } + // The frame itself is not parsed — it is only the signal + // that there is something to read — but the ack is not + // optional: the server bounds unacked updates. + conn.send(&msg_ack()).await?; + if !in_flight { + conn.send(&msg_term_since(READ, id, seq, col, max_bytes, 0)) + .await?; + in_flight = true; + } + } + S2C_TERM_OUTPUT => { + let Some(reply) = blit_remote::journal::parse_s2c_term_output(&data) else { + continue; + }; + if reply.nonce != READ { + continue; + } + in_flight = false; + if reply.status != STATUS_OK { + return Err(format!("pty {id}: {}", status_text(reply.status))); + } + (seq, col) = (reply.next_seq, reply.next_col); + pending.push_str(&reply.text); + if let Some(line) = first_match(&mut pending, re, PENDING_MAX) { + println!("{line}"); + return Ok(0); + } + if reply.flags & OUTPUT_TRUNCATED != 0 { + conn.send(&msg_term_since(READ, id, seq, col, max_bytes, 0)) + .await?; + in_flight = true; + continue; + } + if let Some((status, reason)) = exited { + println!("{}", format_exit(status, reason)); + return Ok(exit_code_from_status(status)); + } + } + S2C_EXITED => { + if let Some(ServerMsg::Exited { + pty_id, + exit_status, + reason, + }) = parse_server_msg(&data) + && pty_id == id + { + exited = Some((exit_status, reason)); + // The process's last words can still match, so drain + // before reporting the exit. + if !in_flight { + conn.send(&msg_term_since(READ, id, seq, col, max_bytes, 0)) + .await?; + in_flight = true; + } + } + } + _ => {} + } + } +} - conn.send(&msg_subscribe(id)).await?; +/// The pre-journal pattern wait: re-scan the whole grid on every update. +/// +/// Kept only for servers without `FEATURE_TERM_JOURNAL`, where there is no +/// cursor to anchor against. It can match text that was on screen before the +/// wait started. +async fn wait_for_pattern_on_grid( + mut conn: AgentConn, + id: u16, + re: ®ex::Regex, + deadline: tokio::time::Instant, +) -> Result { + conn.send(&msg_subscribe(id)).await?; - let mut state = TerminalState::new(0, 0); + let mut state = TerminalState::new(0, 0); - loop { - let data = match conn.recv_deadline(deadline).await { - Ok(d) => d, - Err(e) if e == "timeout" => { - eprintln!("blit: timed out waiting for pty {id}"); - return Ok(124); - } - Err(e) => return Err(e), - }; - if data.is_empty() { - continue; + loop { + let data = match conn.recv_deadline(deadline).await { + Ok(d) => d, + Err(e) if e == "timeout" => { + eprintln!("blit: timed out waiting for pty {id}"); + return Ok(124); } - match data[0] { - S2C_UPDATE if data.len() >= 3 => { - let pid = u16::from_le_bytes([data[1], data[2]]); - if pid == id { - state.feed_compressed(&data[3..]); - conn.send(&msg_ack()).await?; - let text = state.get_all_text(); - for line in text.lines() { - if re.is_match(line) { - println!("{line}"); - return Ok(0); - } - } - if let Some(&status) = conn.exited.get(&id) { - let code = exit_code_from_status(status); - println!("{}", format_exit_status(status)); - return Ok(code); + Err(e) => return Err(e), + }; + if data.is_empty() { + continue; + } + match data[0] { + S2C_UPDATE if data.len() >= 3 => { + let pid = u16::from_le_bytes([data[1], data[2]]); + if pid == id { + state.feed_compressed(&data[3..]); + conn.send(&msg_ack()).await?; + let text = state.get_all_text(); + for line in text.lines() { + if re.is_match(line) { + println!("{line}"); + return Ok(0); } } - } - S2C_EXITED => { - if let Some(ServerMsg::Exited { - pty_id, - exit_status, - reason, - }) = parse_server_msg(&data) - && pty_id == id - { - let code = exit_code_from_status(exit_status); - println!("{}", format_exit(exit_status, reason)); + if let Some(&status) = conn.exited.get(&id) { + let code = exit_code_from_status(status); + println!("{}", format_exit_status(status)); return Ok(code); } } - _ => {} } + S2C_EXITED => { + if let Some(ServerMsg::Exited { + pty_id, + exit_status, + reason, + }) = parse_server_msg(&data) + && pty_id == id + { + let code = exit_code_from_status(exit_status); + println!("{}", format_exit(exit_status, reason)); + return Ok(code); + } + } + _ => {} } + } +} + +pub async fn cmd_wait( + transport: Transport, + id: u16, + timeout_secs: u64, + pattern: Option, +) -> Result { + let mut conn = AgentConn::connect(transport).await?; + + if !conn.has_pty(id) { + return Err(format!("pty {id} not found")); + } + + let deadline = tokio::time::Instant::now() + std::time::Duration::from_secs(timeout_secs); + + if let Some(ref pat) = pattern { + let re = regex::Regex::new(pat).map_err(|e| format!("invalid pattern: {e}"))?; + if conn.features & blit_remote::journal::FEATURE_TERM_JOURNAL != 0 { + return wait_for_pattern(conn, id, &re, deadline).await; + } + // Pre-journal server: no cursor to anchor against, so fall back to + // re-scanning the grid. That can match text that was already there + // when the wait began — the bug this command's cursor path fixes. + eprintln!("blit: server has no output cursor; --pattern may match text already on screen"); + return wait_for_pattern_on_grid(conn, id, &re, deadline).await; } else { if let Some(&status) = conn.exited.get(&id) { let code = exit_code_from_status(status); @@ -1075,11 +1281,21 @@ pub async fn cmd_surfaces(transport: Transport) -> Result<(), String> { } if data[0] == S2C_SURFACE_LIST { if let Some(ServerMsg::SurfaceList { entries }) = parse_server_msg(&data) { - println!("ID\tTITLE\tSIZE\tAPP_ID"); + // ORIGIN is what the socket said; APP_ID is what the app said + // about itself. Keeping both columns makes the difference + // visible, which is the point of having stamped identity at + // all — a blank ORIGIN means the surface came in on the shared + // socket and nothing is known. + println!("ID\tTITLE\tSIZE\tAPP_ID\tORIGIN"); for e in &entries { + let origin = conn + .surface_origins + .get(&e.surface_id) + .map(String::as_str) + .unwrap_or("-"); println!( - "{}\t{}\t{}x{}\t{}", - e.surface_id, e.title, e.width, e.height, e.app_id + "{}\t{}\t{}x{}\t{}\t{}", + e.surface_id, e.title, e.width, e.height, e.app_id, origin ); } } @@ -2126,6 +2342,40 @@ mod tests { .await; } + /// Read the next `C2S_TERM_SINCE`, skipping the acks a subscriber + /// sends along the way. + async fn expect_since(&mut self) -> blit_remote::journal::SinceRequest { + loop { + let data = self.recv().await.expect("client went quiet"); + if data[0] == blit_remote::C2S_ACK { + continue; + } + break blit_remote::journal::parse_term_since(&data) + .unwrap_or_else(|| panic!("wanted TERM_SINCE, got {:#04x}", data[0])); + } + } + + /// Answer a `C2S_TERM_SINCE` with `text` and the cursor after it. + async fn answer_since( + &mut self, + req: &blit_remote::journal::SinceRequest, + next_seq: u64, + text: &str, + ) { + let msg = blit_remote::journal::msg_s2c_term_output( + req.nonce, + req.pty_id, + STATUS_OK, + 0, + req.from_seq, + req.from_col, + next_seq, + 0, + text, + ); + write_frame(&mut self.writer, &msg).await; + } + async fn send_text( &mut self, nonce: u16, @@ -2178,6 +2428,7 @@ mod tests { kind: blit_remote::CLIENT_SUBSCRIPTION_FS, id: 3, }], + origin: None, }, blit_remote::ClientListEntry { client_id: 7, @@ -2187,6 +2438,7 @@ mod tests { terminals: vec![], surfaces: vec![], subscriptions: vec![], + origin: None, }, ], ), @@ -2800,6 +3052,149 @@ mod tests { mock.await.unwrap(); } + // ── wait --pattern, cursor path ────────────────────────────────────── + + /// The bug this fixes: `--pattern` promised "produced after the wait + /// began", but the grid scan matched whatever was already on screen. + #[tokio::test] + async fn wait_pattern_ignores_text_that_was_already_on_screen() { + let (client, server) = tokio::net::UnixStream::pair().unwrap(); + + let mock = tokio::spawn(async move { + let mut mock = MockServer::new(server); + // The pattern is on screen *before* the wait starts. + mock.add_pty(1, "build", "make", false, "BUILD SUCCESS"); + mock.send_initial_burst_with_features(blit_remote::journal::FEATURE_TERM_JOURNAL) + .await; + + let probe = mock.expect_since().await; + assert_eq!(probe.flags, blit_remote::journal::SINCE_PROBE); + mock.answer_since(&probe, 100, "").await; + + let subscribe = mock.recv().await.unwrap(); + assert_eq!(subscribe[0], blit_remote::C2S_SUBSCRIBE); + + let first = mock.expect_since().await; + assert_eq!(first.from_seq, 100); + mock.answer_since(&first, 100, "").await; + + // An update whose grid holds the pattern, but no new output. + mock.send_update_for(1).await; + let after_update = mock.expect_since().await; + mock.answer_since(&after_update, 100, "").await; + + mock.send_exited(1, 7).await; + let drain = mock.expect_since().await; + mock.answer_since(&drain, 100, "").await; + }); + + let transport = Transport::Unix(client); + let result = cmd_wait(transport, 1, 5, Some("BUILD (SUCCESS|FAILURE)".to_string())).await; + // 7, not 0: the wait ended at the exit, not at the stale line. + assert_eq!(result.unwrap(), 7); + + mock.await.unwrap(); + } + + #[tokio::test] + async fn wait_pattern_matches_output_produced_after_the_wait_began() { + let (client, server) = tokio::net::UnixStream::pair().unwrap(); + + let mock = tokio::spawn(async move { + let mut mock = MockServer::new(server); + mock.add_pty(1, "build", "make", false, "compiling..."); + mock.send_initial_burst_with_features(blit_remote::journal::FEATURE_TERM_JOURNAL) + .await; + + let probe = mock.expect_since().await; + mock.answer_since(&probe, 40, "").await; + assert_eq!(mock.recv().await.unwrap()[0], blit_remote::C2S_SUBSCRIBE); + let first = mock.expect_since().await; + mock.answer_since(&first, 40, "").await; + + mock.send_update_for(1).await; + let read = mock.expect_since().await; + assert_eq!(read.from_seq, 40); + mock.answer_since(&read, 42, "linking\nBUILD FAILURE\n") + .await; + }); + + let transport = Transport::Unix(client); + let result = cmd_wait(transport, 1, 5, Some("BUILD (SUCCESS|FAILURE)".to_string())).await; + assert_eq!(result.unwrap(), 0); + + mock.await.unwrap(); + } + + /// A line delivered in pieces still matches once, when it is complete. + #[tokio::test] + async fn wait_pattern_matches_a_line_split_across_reads() { + let (client, server) = tokio::net::UnixStream::pair().unwrap(); + + let mock = tokio::spawn(async move { + let mut mock = MockServer::new(server); + mock.add_pty(1, "build", "make", false, ""); + mock.send_initial_burst_with_features(blit_remote::journal::FEATURE_TERM_JOURNAL) + .await; + + let probe = mock.expect_since().await; + mock.answer_since(&probe, 10, "").await; + assert_eq!(mock.recv().await.unwrap()[0], blit_remote::C2S_SUBSCRIBE); + let first = mock.expect_since().await; + mock.answer_since(&first, 10, "").await; + + mock.send_update_for(1).await; + let half = mock.expect_since().await; + mock.answer_since(&half, 10, "BUILD FAIL").await; + + mock.send_update_for(1).await; + let rest = mock.expect_since().await; + mock.answer_since(&rest, 11, "URE\n").await; + }); + + let transport = Transport::Unix(client); + let result = cmd_wait(transport, 1, 5, Some("^BUILD FAILURE$".to_string())).await; + assert_eq!(result.unwrap(), 0); + + mock.await.unwrap(); + } + + #[test] + fn first_match_scans_only_what_is_new() { + let re = regex::Regex::new("ready").unwrap(); + let mut pending = String::from("starting\nnot yet\npartial"); + assert_eq!(first_match(&mut pending, &re, 1024), None); + // Complete lines are consumed; the partial tail is kept to be + // re-examined when the rest of it arrives. + assert_eq!(pending, "partial"); + pending.push_str(" ready\n"); + assert_eq!( + first_match(&mut pending, &re, 1024).as_deref(), + Some("partial ready") + ); + } + + #[test] + fn first_match_matches_a_line_still_being_written() { + let re = regex::Regex::new("password:").unwrap(); + let mut pending = String::from("password:"); + assert_eq!( + first_match(&mut pending, &re, 1024).as_deref(), + Some("password:") + ); + } + + #[test] + fn first_match_bounds_a_line_that_never_ends() { + let re = regex::Regex::new("never").unwrap(); + let mut pending = String::new(); + for _ in 0..100 { + pending.push_str(&"x".repeat(100)); + assert_eq!(first_match(&mut pending, &re, 256), None); + assert!(pending.len() <= 256, "pending grew to {}", pending.len()); + } + } + #[tokio::test] async fn test_wait_pattern_match() { let (client, server) = tokio::net::UnixStream::pair().unwrap(); @@ -2807,6 +3202,8 @@ mod tests { let mock = tokio::spawn(async move { let mut mock = MockServer::new(server); mock.add_pty(1, "build", "make", false, "BUILD SUCCESS"); + // No journal feature bit: the legacy grid scan, which matches + // text that was already on screen. mock.send_initial_burst().await; let data = mock.recv().await.unwrap(); diff --git a/crates/cli/src/cli.rs b/crates/cli/src/cli.rs index 23a912fc..669a1fc7 100644 --- a/crates/cli/src/cli.rs +++ b/crates/cli/src/cli.rs @@ -723,24 +723,42 @@ pub enum TerminalCommand { /// /// Without position flags, prints everything. Use --from-beginning or /// --from-end to set a starting offset, and --limit to cap the output. + /// + /// --since CURSOR instead reads only what was appended since a cursor + /// and prints the next cursor on stderr, so a loop pulls each byte once: + /// c=$(blit terminal history 3 --since now 2>&1 >/dev/null | cut -d' ' -f2) + /// blit terminal history 3 --since "$c" + /// CURSOR is SEQ, SEQ:COL, `now` (no text, current cursor) or `start`. History { /// Terminal ID id: u16, /// Start N lines from the top (oldest = 0) - #[arg(long, conflicts_with = "from_end")] + #[arg(long, conflicts_with_all = ["from_end", "since"])] from_start: Option, /// Start N lines from the bottom (newest = 0) - #[arg(long, conflicts_with = "from_start")] + #[arg(long, conflicts_with_all = ["from_start", "since"])] from_end: Option, /// Maximum number of lines to return - #[arg(long)] + #[arg(long, conflicts_with = "since")] limit: Option, + /// Read only what is new since this cursor (SEQ[:COL], now, start) + #[arg(long, value_name = "CURSOR")] + since: Option, + + /// Cap a --since read; the reply says where to continue from + #[arg(long, value_name = "BYTES", requires = "since")] + max_bytes: Option, + + /// JSON output (with --since: text plus the next cursor) + #[arg(long, requires = "since")] + json: bool, + /// Include ANSI color/style escape sequences in output - #[arg(long)] + #[arg(long, conflicts_with = "since")] ansi: bool, /// Resize to this many rows before capturing @@ -752,6 +770,59 @@ pub enum TerminalCommand { cols: Option, }, + /// List the commands a terminal has run (needs OSC 133 shell integration). + /// + /// TSV: INDEX, STATUS, EXIT, MS, START_SEQ, END_SEQ, COMMAND. Prints the + /// most recent commands by default. Indices are stable and never reused, + /// so one can be fed back to `blit terminal output`. + /// + /// Nothing is recorded unless the shell emits OSC 133 semantic prompts — + /// see docs/shell-integration.md for the one-line hook. + Journal { + /// Terminal ID + id: u16, + + /// Start at this command index instead of the newest ones + #[arg(long, value_name = "INDEX")] + from: Option, + + /// Maximum number of records + #[arg(long, default_value_t = crate::journal::JOURNAL_LIMIT)] + limit: u16, + + /// One JSON object per record + #[arg(long)] + json: bool, + }, + + /// Print one command's output (needs OSC 133 shell integration). + /// + /// Defaults to the newest command. With --wait, blocks server-side until + /// the command finishes and exits with its status (124 if the wait timed + /// out), which is how to run something in a live shell and collect the + /// result: + /// blit terminal send 3 'cargo test\n' + /// blit terminal output 3 --wait 600 + Output { + /// Terminal ID + id: u16, + + /// Command index from `blit terminal journal` (default: newest) + index: Option, + + /// Block up to SECONDS for the command to finish first + #[arg(long, value_name = "SECONDS")] + wait: Option, + + /// Cap the output; the reply says where to continue from + #[arg(long, value_name = "BYTES", default_value_t = crate::journal::OUTPUT_MAX_BYTES)] + max_bytes: u32, + + /// JSON: the record plus its output text + #[arg(long)] + json: bool, + }, + /// Ripgrep-compatible search over terminals' backlog + viewport. /// /// Each terminal is treated as a "file". Trailing IDs pick specific terminals diff --git a/crates/cli/src/extension.rs b/crates/cli/src/extension.rs index 15398c24..d60d4cfb 100644 --- a/crates/cli/src/extension.rs +++ b/crates/cli/src/extension.rs @@ -69,12 +69,8 @@ pub struct RunArgs { pub detach: bool, /// Store an enabled, desired-running definition (implies --detach) - #[arg(long, requires = "name")] - pub persist: bool, - - /// Descriptive name, or the unique durable name with --persist #[arg(long)] - pub name: Option, + pub persist: bool, /// Attempt restart policy #[arg(long, value_enum, default_value_t)] @@ -84,9 +80,18 @@ pub struct RunArgs { #[arg(long)] pub json: bool, - /// WebAssembly module followed by UTF-8 arguments passed verbatim + // Positional, in the same place `update` takes it, so the two commands read + // alike. It was an optional `--name` flag, which meant `--persist` had to + // declare `requires = "name"` and the help had to explain when a name + // mattered; every extension has one worth printing, so requiring it is + // simpler than describing the exception. + /// A label, or the unique durable name under --persist + pub name: String, + + /// WebAssembly module (a path or an https:// URL) followed by UTF-8 + /// arguments passed verbatim #[arg( - value_names = ["FILE", "ARGS"], + value_names = ["MODULE", "ARGS"], num_args = 1.., trailing_var_arg = true, allow_hyphen_values = true @@ -107,9 +112,10 @@ pub struct UpdateArgs { /// Exact persistent extension name pub name: String, - /// Replacement WebAssembly module followed by UTF-8 arguments passed verbatim + /// Replacement WebAssembly module (a path or an https:// URL) followed + /// by UTF-8 arguments passed verbatim #[arg( - value_names = ["FILE", "ARGS"], + value_names = ["MODULE", "ARGS"], num_args = 1.., trailing_var_arg = true, allow_hyphen_values = true @@ -251,23 +257,24 @@ pub(crate) async fn complete_advertised_commands( } async fn run(client: &mut Client, args: RunArgs) -> Result { - validate_name(args.name.as_deref().unwrap_or(""))?; + validate_name(&args.name)?; let (file, extension_args) = split_invocation(args.invocation)?; validate_args(&extension_args)?; - let module = ModuleObject::read(&file)?; + let mut module = ModuleCache::new(file); + let hash = module.hash().await?; let detached = args.detach || args.persist; let mut flags = u8::from(detached) * EXT_RUN_DETACH; if args.persist { flags |= EXT_RUN_PERSIST; } - let name = args.name.as_deref().unwrap_or(""); + let name = args.name.as_str(); let status = client .run_request(RunRequest { flags, restart: args.restart.wire(), expected_id: 0, expected_revision: 0, - hash: module.hash, + hash, name, args: &extension_args, }) @@ -279,7 +286,7 @@ async fn run(client: &mut Client, args: RunArgs) -> Result { return Err("server returned a successful run without an extension ID".into()); } if status.phase == PHASE_NEED_OBJECT { - let _ = client.upload(&module).await?; + let _ = client.upload(module.get().await?).await?; } if args.json { render_status(&status, true, "status"); @@ -289,7 +296,7 @@ async fn run(client: &mut Client, args: RunArgs) -> Result { } else { FollowMode::Owned }; - follow(client, id, status, args.json, mode, Some(&module)).await + follow(client, id, status, args.json, mode, Some(&mut module)).await } async fn update(client: &mut Client, args: UpdateArgs) -> Result { @@ -297,7 +304,8 @@ async fn update(client: &mut Client, args: UpdateArgs) -> Result { validate_name(name)?; let (file, extension_args) = split_invocation(args.invocation)?; validate_args(&extension_args)?; - let module = ModuleObject::read(&file)?; + let mut module = ModuleCache::new(file); + let hash = module.hash().await?; let records = client.list().await?; let record = find_named(&records, name)?; if record.flags & EXT_FLAG_PERSIST == 0 { @@ -314,7 +322,7 @@ async fn update(client: &mut Client, args: UpdateArgs) -> Result { restart, expected_id, expected_revision, - hash: module.hash, + hash, name, args: &extension_args, }) @@ -325,7 +333,7 @@ async fn update(client: &mut Client, args: UpdateArgs) -> Result { return Ok(0); } - match client.upload(&module).await? { + match client.upload(module.get().await?).await? { UploadOutcome::Available => { // Uploading an update only primes the CAS. Recheck the exact // tuple before retrying; never adopt a concurrent revision. @@ -377,7 +385,7 @@ async fn follow( mut lifecycle: StatusRecord, json: bool, mode: FollowMode, - module: Option<&ModuleObject>, + mut module: Option<&mut ModuleCache>, ) -> Result { let mut last_exit: Option = None; let replay_snapshot_through = match mode { @@ -465,9 +473,9 @@ async fn follow( render_status(&lifecycle, true, "status"); } if lifecycle.phase == PHASE_NEED_OBJECT - && let Some(module) = module + && let Some(module) = module.as_deref_mut() { - let _ = client.upload(module).await?; + let _ = client.upload(module.get().await?).await?; } } } @@ -478,9 +486,9 @@ async fn follow( render_status(&lifecycle, true, "status"); } if lifecycle.phase == PHASE_NEED_OBJECT - && let Some(module) = module + && let Some(module) = module.as_deref_mut() { - let _ = client.upload(module).await?; + let _ = client.upload(module.get().await?).await?; } } wire::Message::InfoStatus(status) if status.extension_id == extension_id => { @@ -493,9 +501,9 @@ async fn follow( if is_current { lifecycle = status; if lifecycle.phase == PHASE_NEED_OBJECT - && let Some(module) = module + && let Some(module) = module.as_deref_mut() { - let _ = client.upload(module).await?; + let _ = client.upload(module.get().await?).await?; } } } @@ -780,12 +788,12 @@ fn validate_name(name: &str) -> Result<(), String> { Ok(()) } -fn split_invocation(invocation: Vec) -> Result<(PathBuf, Vec), String> { +fn split_invocation(invocation: Vec) -> Result<(ModuleSource, Vec), String> { let mut invocation = invocation.into_iter(); let file = invocation .next() - .map(PathBuf::from) - .ok_or_else(|| "missing WebAssembly module FILE".to_string())?; + .ok_or_else(|| "missing WebAssembly module MODULE".to_string()) + .and_then(ModuleSource::parse)?; let args = invocation .map(|arg| { arg.into_string() @@ -849,10 +857,12 @@ fn forced_name(text: &str) -> Result<&str, String> { async fn resolve_selector(client: &mut Client, text: &str) -> Result { match selector(text)? { Selector::Id(id) => Ok(id), - Selector::Name(name) => Ok(find_named(&client.list().await?, name)?.extension_id), + Selector::Name(name) => Ok(find_selectable(&client.list().await?, name)?.extension_id), } } +/// The durable definition called `name`. Only a persistent extension has one: +/// there, the name is the identity that `update` and `remove` act on. fn find_named<'a>( records: &'a [ExtensionRecord], name: &str, @@ -863,12 +873,234 @@ fn find_named<'a>( .ok_or_else(|| format!("extension name not found: {name}")) } +/// Anything a selector may point at: the durable definition when there is one, +/// otherwise a transient attempt whose descriptive name is unambiguous. +/// +/// A transient `--name` is a label rather than an identity, so it resolves +/// only while it happens to be unique. Refusing it outright reads as "no such +/// extension" for a name `ext list` is displaying right there. +fn find_selectable<'a>( + records: &'a [ExtensionRecord], + name: &str, +) -> Result<&'a ExtensionRecord, String> { + if let Ok(persistent) = find_named(records, name) { + return Ok(persistent); + } + let mut matches = records.iter().filter(|record| record.name == name); + match (matches.next(), matches.next()) { + (Some(only), None) => Ok(only), + (Some(_), Some(_)) => Err(format!( + "{name} is the descriptive name of more than one extension; select it by id:" + )), + _ => Err(format!("extension name not found: {name}")), + } +} + +/// Where the module bytes come from. The server never learns either form: +/// `EXT_RUN` carries the BLAKE3 digest, so a locator is purely a client-side +/// convenience that ends in the same content-addressed admission path. +/// +/// A URL may pin that digest in its fragment — +/// `https://install.blit.sh/systemd#<64-hex>` — which names one exact object +/// in one argument. The fragment never reaches the origin server: it is a +/// client-side assertion about the bytes, not part of the request. +#[derive(Clone, Debug, Eq, PartialEq)] +enum ModuleSource { + File(PathBuf), + Url { url: String, pin: Option<[u8; 32]> }, +} + +/// How long a module download may take before the command gives up. +const FETCH_TIMEOUT: Duration = Duration::from_secs(30); + +fn parse_digest(text: &str) -> Result<[u8; 32], String> { + if text.len() != 64 || !text.bytes().all(|byte| byte.is_ascii_hexdigit()) { + return Err(format!( + "expected a 64-hex-digit BLAKE3 digest, got {text:?}" + )); + } + let mut digest = [0u8; 32]; + for (index, byte) in digest.iter_mut().enumerate() { + *byte = u8::from_str_radix(&text[index * 2..index * 2 + 2], 16) + .map_err(|error| format!("invalid BLAKE3 digest: {error}"))?; + } + Ok(digest) +} + +fn format_digest(digest: &[u8; 32]) -> String { + digest.iter().fold(String::new(), |mut text, byte| { + use fmt::Write as _; + let _ = write!(text, "{byte:02x}"); + text + }) +} + +impl ModuleSource { + fn parse(token: OsString) -> Result { + let text = token.to_string_lossy(); + if !text.starts_with("https://") && !text.starts_with("http://") { + return Ok(Self::File(PathBuf::from(token))); + } + let mut url = + reqwest::Url::parse(&text).map_err(|error| format!("cannot parse {text}: {error}"))?; + let pin = match url.fragment() { + Some(fragment) => Some( + parse_digest(fragment) + .map_err(|error| format!("{text}: pinned digest is invalid: {error}"))?, + ), + None => None, + }; + url.set_fragment(None); + Ok(Self::Url { + url: url.into(), + pin, + }) + } + + /// The digest this source is already known to have, if any. + /// + /// A pinned URL can be admitted before a single byte is fetched: the + /// server answers `EXT_RUN` from its object cache, and the download only + /// happens if it asks for one. + fn pinned(&self) -> Option<[u8; 32]> { + match self { + Self::Url { pin, .. } => *pin, + Self::File(_) => None, + } + } + + async fn load(&self) -> Result { + let module = match self { + Self::File(path) => ModuleObject::read(path)?, + Self::Url { url, .. } => ModuleObject::fetch(url).await?, + }; + if let Some(pin) = self.pinned() + && module.hash != pin + { + return Err(format!( + "digest mismatch: pinned {} but the bytes hash to {}", + format_digest(&pin), + format_digest(&module.hash) + )); + } + Ok(module) + } +} + +/// One module, fetched at most once however many times it is needed. +/// +/// The server can ask for the bytes again mid-run — an upload can expire, and +/// an unpinned object can be evicted — so a run that started from a pinned URL +/// still has to be able to produce them later. +struct ModuleCache { + source: ModuleSource, + loaded: Option, +} + +impl ModuleCache { + fn new(source: ModuleSource) -> Self { + Self { + source, + loaded: None, + } + } + + fn pinned(&self) -> Option<[u8; 32]> { + self.source.pinned() + } + + async fn get(&mut self) -> Result<&ModuleObject, String> { + if self.loaded.is_none() { + self.loaded = Some(self.source.load().await?); + } + Ok(self.loaded.as_ref().expect("just loaded")) + } + + /// The digest to put in `EXT_RUN`: the pin when there is one, otherwise + /// whatever the bytes turn out to hash to. + async fn hash(&mut self) -> Result<[u8; 32], String> { + match self.pinned() { + Some(pin) => Ok(pin), + None => Ok(self.get().await?.hash), + } + } +} + +/// Cleartext carries no integrity, and the digest this client computes comes +/// from the very bytes an attacker would have substituted, so plain HTTP is +/// only honest when it cannot leave the machine. +fn loopback_url(url: &reqwest::Url) -> bool { + match url.host() { + Some(url::Host::Domain(name)) => name == "localhost", + Some(url::Host::Ipv4(address)) => address.is_loopback(), + Some(url::Host::Ipv6(address)) => address.is_loopback(), + None => false, + } +} + struct ModuleObject { bytes: Vec, hash: [u8; 32], } impl ModuleObject { + async fn fetch(url: &str) -> Result { + let parsed = + reqwest::Url::parse(url).map_err(|error| format!("cannot parse {url}: {error}"))?; + if parsed.scheme() == "http" && !loopback_url(&parsed) { + return Err(format!( + "{url}: refusing plain HTTP to a non-loopback host; use https://" + )); + } + let client = reqwest::Client::builder() + .timeout(FETCH_TIMEOUT) + .redirect(reqwest::redirect::Policy::limited(5)) + .user_agent(concat!("blit/", env!("CARGO_PKG_VERSION"))) + .build() + .map_err(|error| format!("cannot build an HTTP client: {error}"))?; + let mut response = client + .get(parsed) + .send() + .await + .map_err(|error| format!("cannot fetch {url}: {error}"))? + .error_for_status() + .map_err(|error| format!("cannot fetch {url}: {error}"))?; + // A redirect chain must not walk out of TLS on the way to the bytes. + if response.url().scheme() == "http" && !loopback_url(response.url()) { + return Err(format!( + "{url}: redirected to plain HTTP ({}); refusing", + response.url() + )); + } + if let Some(length) = response.content_length() + && length > EXT_MAX_MODULE + { + return Err(format!( + "{url} declares {length} bytes, over the {EXT_MAX_MODULE}-byte extension limit" + )); + } + // Stream with the cap enforced per chunk: a declared length is a hint, + // not a promise, and this runs against whatever the URL names. + let mut bytes = Vec::new(); + while let Some(chunk) = response + .chunk() + .await + .map_err(|error| format!("cannot read {url}: {error}"))? + { + if bytes.len() as u64 + chunk.len() as u64 > EXT_MAX_MODULE { + return Err(format!( + "{url} exceeds the {EXT_MAX_MODULE}-byte extension limit" + )); + } + bytes.extend_from_slice(&chunk); + } + if bytes.is_empty() { + return Err(format!("{url} returned no bytes")); + } + let hash = *blake3::hash(&bytes).as_bytes(); + Ok(Self { bytes, hash }) + } + fn read(path: &Path) -> Result { let metadata = std::fs::metadata(path) .map_err(|e| format!("cannot inspect {}: {e}", path.display()))?; @@ -1612,6 +1844,7 @@ mod tests { "--detach", "--restart", "on-failure", + "labelled", "module.wasm", "--on", "guest-target", @@ -1632,6 +1865,7 @@ mod tests { "--detach", "--restart", "on-failure", + "labelled", "module.wasm", "--on", "guest-target", @@ -1664,7 +1898,7 @@ mod tests { let first = split_invocation(first.invocation).unwrap(); let second = split_invocation(second.invocation).unwrap(); assert_eq!(second, first); - assert_eq!(first.0, PathBuf::from("module.wasm")); + assert_eq!(first.0, ModuleSource::File(PathBuf::from("module.wasm"))); assert_eq!( first.1, [ @@ -1712,7 +1946,7 @@ mod tests { assert!(args.json); assert_eq!(args.restart, Some(RestartPolicy::OnFailure)); let (file, args) = split_invocation(args.invocation).unwrap(); - assert_eq!(file, PathBuf::from("module.wasm")); + assert_eq!(file, ModuleSource::File(PathBuf::from("module.wasm"))); assert_eq!( args, [ @@ -1727,6 +1961,125 @@ mod tests { ); } + #[test] + fn module_sources_split_urls_from_paths() { + assert_eq!( + ModuleSource::parse(OsString::from("https://install.blit.sh/systemd")).unwrap(), + ModuleSource::Url { + url: "https://install.blit.sh/systemd".into(), + pin: None + } + ); + // Only these two schemes are locators; everything else is a path, so a + // Windows drive letter and a relative name keep working. + for path in ["module.wasm", "./module.wasm", "C:\\modules\\a.wasm"] { + assert_eq!( + ModuleSource::parse(OsString::from(path)).unwrap(), + ModuleSource::File(PathBuf::from(path)) + ); + } + } + + #[test] + fn url_fragments_pin_one_exact_object() { + let digest = "1a3baedf416f2b0f9b6cd683a01d8408a1c6928ba698c0533fdd81aca8fc7e2c"; + let source = ModuleSource::parse(OsString::from(format!( + "https://install.blit.sh/systemd#{digest}" + ))) + .unwrap(); + // The fragment is a client-side assertion, so it must not survive into + // the request the origin server sees. + assert_eq!( + source, + ModuleSource::Url { + url: "https://install.blit.sh/systemd".into(), + pin: Some(parse_digest(digest).unwrap()), + } + ); + assert_eq!(format_digest(&source.pinned().unwrap()), digest); + assert_eq!( + ModuleSource::parse(OsString::from("https://install.blit.sh/systemd")) + .unwrap() + .pinned(), + None + ); + for bad in [ + "#", + "#deadbeef", + "#zz", + "#1a3baedf416f2b0f9b6cd683a01d8408a1c6928ba6", + ] { + assert!( + ModuleSource::parse(OsString::from(format!("https://install.blit.sh/x{bad}"))) + .is_err() + ); + } + // A path is never reinterpreted: '#' is a legal filename byte. + assert_eq!( + ModuleSource::parse(OsString::from("./mod#1.wasm")).unwrap(), + ModuleSource::File(PathBuf::from("./mod#1.wasm")) + ); + } + + #[test] + fn plain_http_is_only_trusted_on_loopback() { + let loopback = ["http://localhost:8080/a.wasm", "http://127.0.0.1/a.wasm"]; + for url in loopback { + assert!(loopback_url(&reqwest::Url::parse(url).unwrap())); + } + for url in ["http://install.blit.sh/a.wasm", "http://10.0.0.1/a.wasm"] { + assert!(!loopback_url(&reqwest::Url::parse(url).unwrap())); + } + } + + #[test] + fn selectors_reach_transient_attempts_by_their_label() { + let record = |extension_id: u64, name: &str, flags: u8| ExtensionRecord { + extension_id, + definition_revision: 1, + phase: PHASE_RUNNING, + flags, + restart: EXT_RESTART_NEVER, + attempt: 1, + last_running_attempt: 1, + task_id: 0, + output_sequence: 0, + next_start_unix_ms: 0, + hash: [0; 32], + name: String::from(name), + }; + // A durable definition owns its name even while it is stopped. + let mixed = [ + record(1, "systemd", EXT_FLAG_DETACH), + record(2, "systemd", EXT_FLAG_PERSIST), + ]; + assert_eq!(find_selectable(&mixed, "systemd").unwrap().extension_id, 2); + + // With no definition, an unambiguous label still resolves: `ext list` + // prints that name, so refusing it reads as "no such extension". + let transient = [record(7, "systemd", EXT_FLAG_DETACH)]; + assert_eq!( + find_selectable(&transient, "systemd").unwrap().extension_id, + 7 + ); + assert!(find_named(&transient, "systemd").is_err()); + + let twins = [ + record(7, "systemd", EXT_FLAG_DETACH), + record(8, "systemd", EXT_FLAG_DETACH), + ]; + assert!( + find_selectable(&twins, "systemd") + .unwrap_err() + .contains("more than one") + ); + assert!( + find_selectable(&twins, "other") + .unwrap_err() + .contains("not found") + ); + } + #[test] fn selectors_never_guess_bare_hex_as_an_id() { assert!(matches!( diff --git a/crates/cli/src/journal.rs b/crates/cli/src/journal.rs new file mode 100644 index 00000000..18ad7b5a --- /dev/null +++ b/crates/cli/src/journal.rs @@ -0,0 +1,500 @@ +//! `blit terminal journal` / `output` / `history --since` — the per-command +//! journal and the sequence cursor (docs/design/term-journal.md). +//! +//! Three questions an agent driving a long-lived shell actually asks: what +//! commands has this terminal run, what did one of them print, and what is +//! new since I last looked. All three are answered by index or by cursor, so +//! nothing here pulls the whole scrollback by accident. +//! +//! Everything degrades to an error rather than a hang against a server +//! without `FEATURE_TERM_JOURNAL`: an old server drops the opcode silently +//! and the request would never be answered. + +use blit_remote::journal::{ + CommandRecord, FEATURE_TERM_JOURNAL, JOURNAL_INDEX_LATEST, JOURNAL_TAIL, OUTPUT_ALT_SCREEN, + OUTPUT_EVICTED, OUTPUT_TRUNCATED, OutputReply, RECORD_EVICTED, RECORD_HAS_EXIT, + RECORD_INCOMPLETE, RECORD_NO_COMMAND, RECORD_PTY_EXITED, S2C_TERM_COMMAND, S2C_TERM_JOURNAL, + S2C_TERM_OUTPUT, SINCE_PROBE, msg_term_journal, msg_term_journal_wait, msg_term_output, + msg_term_since, parse_s2c_term_command, parse_s2c_term_journal, parse_s2c_term_output, +}; +use blit_remote::{STATUS_NOT_FOUND, STATUS_OK, status_text}; + +use crate::agent::AgentConn; +use crate::transport::Transport; + +/// Default cap on a single output read. Large enough that an ordinary +/// command comes back whole, small enough that a runaway one does not have +/// to be paged through in a thousand steps. +pub const OUTPUT_MAX_BYTES: u32 = 256 * 1024; + +/// Default number of records `blit terminal journal` prints. +pub const JOURNAL_LIMIT: u16 = 20; + +const NONCE: u16 = 1; + +/// How long to wait for a reply that the server produces synchronously. +fn reply_deadline() -> tokio::time::Instant { + tokio::time::Instant::now() + std::time::Duration::from_secs(10) +} + +/// A cursor as the CLI spells it: `SEQ` or `SEQ:COL`, plus the two words +/// that save a caller from having to know a number at all. +pub(crate) enum Cursor { + /// `now` — ask the server where the terminal currently is, and read + /// nothing. This is how a caller starts following output. + Now, + At(u64, u16), +} + +pub(crate) fn parse_cursor(text: &str) -> Result { + match text { + "now" | "end" => return Ok(Cursor::Now), + "start" | "oldest" => return Ok(Cursor::At(0, 0)), + _ => {} + } + let (seq, col) = match text.split_once(':') { + Some((seq, col)) => (seq, Some(col)), + None => (text, None), + }; + let seq = seq + .parse::() + .map_err(|_| format!("not a cursor: {text} (want SEQ, SEQ:COL, now, or start)"))?; + let col = match col { + Some(col) => col + .parse::() + .map_err(|_| format!("not a column: {col}"))?, + None => 0, + }; + Ok(Cursor::At(seq, col)) +} + +/// The cursor as a client should print it and feed it back in. +pub(crate) fn format_cursor(seq: u64, col: u16) -> String { + format!("{seq}:{col}") +} + +fn require_feature(conn: &AgentConn, id: u16) -> Result<(), String> { + if conn.features & FEATURE_TERM_JOURNAL == 0 { + return Err( + "server has no terminal journal (upgrade blit on the remote, or BLIT_TERM_JOURNAL=0 \ + is set)" + .to_string(), + ); + } + if !conn.has_pty(id) { + return Err(format!("pty {id} not found")); + } + Ok(()) +} + +/// Read replies until the one carrying our nonce arrives. +/// +/// Anything else on the wire — updates for other subscribers, titles, +/// pings — is not ours to interpret here and is dropped. +async fn await_output(conn: &mut AgentConn, nonce: u16) -> Result { + let deadline = reply_deadline(); + loop { + let data = conn.recv_deadline(deadline).await?; + if data.first() != Some(&S2C_TERM_OUTPUT) { + continue; + } + if let Some(reply) = parse_s2c_term_output(&data) + && reply.nonce == nonce + { + return Ok(reply); + } + } +} + +/// Where the terminal is right now, as a cursor. Nothing is read. +pub(crate) async fn probe_cursor( + conn: &mut AgentConn, + id: u16, + nonce: u16, +) -> Result<(u64, u16), String> { + conn.send(&msg_term_since(nonce, id, 0, 0, 0, SINCE_PROBE)) + .await?; + let reply = await_output(conn, nonce).await?; + check_status(reply.status, id)?; + Ok((reply.next_seq, reply.next_col)) +} + +fn check_status(status: u8, id: u16) -> Result<(), String> { + match status { + STATUS_OK => Ok(()), + STATUS_NOT_FOUND => Err(format!("pty {id}: no such terminal or command")), + other => Err(format!("pty {id}: {}", status_text(other))), + } +} + +fn record_status(record: &CommandRecord) -> &'static str { + if record.running() { + "running" + } else if record.flags & RECORD_INCOMPLETE != 0 { + "incomplete" + } else if record.flags & RECORD_HAS_EXIT != 0 { + "exited" + } else { + "done" + } +} + +fn duration_ms(record: &CommandRecord) -> Option { + (record.ended_ms >= record.started_ms && record.started_ms != 0 && record.ended_ms != 0) + .then(|| record.ended_ms - record.started_ms) +} + +fn record_json(record: &CommandRecord) -> serde_json::Value { + serde_json::json!({ + "index": record.index, + "command": record.command, + "status": record_status(record), + "exit": record.exit(), + "running": record.running(), + "start_seq": record.start_seq, + "end_seq": record.end_seq, + "cursor": format_cursor(record.start_seq, 0), + "started_ms": record.started_ms, + "ended_ms": record.ended_ms, + "duration_ms": duration_ms(record), + "command_known": record.flags & RECORD_NO_COMMAND == 0, + "incomplete": record.flags & RECORD_INCOMPLETE != 0, + "evicted": record.flags & RECORD_EVICTED != 0, + "pty_exited": record.flags & RECORD_PTY_EXITED != 0, + }) +} + +fn print_record_tsv(record: &CommandRecord) { + let exit = record.exit().map(|c| c.to_string()); + let duration = duration_ms(record).map(|ms| ms.to_string()); + println!( + "{}\t{}\t{}\t{}\t{}\t{}\t{}", + record.index, + record_status(record), + exit.as_deref().unwrap_or("-"), + duration.as_deref().unwrap_or("-"), + record.start_seq, + record.end_seq, + // Commands can contain anything; keep the row one line. + record.command.replace(['\t', '\n'], " "), + ); +} + +/// `blit terminal journal ID` — the commands this terminal has run. +pub async fn cmd_journal( + transport: Transport, + id: u16, + from: Option, + limit: u16, + json: bool, +) -> Result { + let mut conn = AgentConn::connect(transport).await?; + require_feature(&conn, id)?; + + // Without --from the interesting end is the newest, so the request + // counts back from there rather than paging up from the oldest. + let (from_index, flags) = match from { + Some(index) => (index, 0), + None => (0, JOURNAL_TAIL), + }; + conn.send(&msg_term_journal(NONCE, id, from_index, limit, flags)) + .await?; + + let deadline = reply_deadline(); + loop { + let data = conn.recv_deadline(deadline).await?; + if data.first() != Some(&S2C_TERM_JOURNAL) { + continue; + } + let Some(reply) = parse_s2c_term_journal(&data) else { + continue; + }; + if reply.nonce != NONCE { + continue; + } + check_status(reply.status, id)?; + if json { + for record in &reply.records { + println!("{}", record_json(record)); + } + } else { + println!("INDEX\tSTATUS\tEXIT\tMS\tSTART_SEQ\tEND_SEQ\tCOMMAND"); + for record in &reply.records { + print_record_tsv(record); + } + if reply.records.is_empty() { + // An empty journal is nearly always a missing shell hook + // rather than an idle terminal, and that is not obvious. + eprintln!( + "blit: no commands recorded — does the shell emit OSC 133? \ + See docs/shell-integration.md" + ); + } + } + return Ok(0); + } +} + +/// Wait for a command to finish, returning the record the server settled on. +async fn wait_for_command( + conn: &mut AgentConn, + id: u16, + index: u64, + timeout_secs: u64, +) -> Result { + let timeout_ms = timeout_secs.saturating_mul(1000).min(u32::MAX as u64) as u32; + conn.send(&msg_term_journal_wait(NONCE, id, index, timeout_ms)) + .await?; + // One second of slack over the server's own timeout: the server always + // answers, and a client that gives up first would report a timeout that + // did not happen. + let deadline = tokio::time::Instant::now() + + std::time::Duration::from_secs(timeout_secs.saturating_add(1)); + loop { + let data = conn.recv_deadline(deadline).await.map_err(|e| { + if e == "timeout" { + format!("pty {id}: server did not answer the wait") + } else { + e + } + })?; + if data.first() != Some(&S2C_TERM_COMMAND) { + continue; + } + let Some((nonce, _pty, status, record)) = parse_s2c_term_command(&data) else { + continue; + }; + if nonce != NONCE { + continue; + } + if status == STATUS_NOT_FOUND { + return Err(format!( + "pty {id}: no such command (nothing running, or it was evicted)" + )); + } + check_status(status, id)?; + return Ok(record); + } +} + +/// `blit terminal output ID [INDEX]` — one command's output. +/// +/// With `--wait` this is the closest thing to "run it and give me the +/// result": block until the command finishes, print what it printed, and +/// exit with its status. +pub async fn cmd_output( + transport: Transport, + id: u16, + index: Option, + wait: Option, + max_bytes: u32, + json: bool, +) -> Result { + let mut conn = AgentConn::connect(transport).await?; + require_feature(&conn, id)?; + + let index = index.unwrap_or(JOURNAL_INDEX_LATEST); + let record = match wait { + Some(timeout) => Some(wait_for_command(&mut conn, id, index, timeout).await?), + None => None, + }; + // The wait latches whichever command it attached to, so fetch that one + // rather than re-resolving "latest" against a shell that has moved on. + let index = record.as_ref().map(|r| r.index).unwrap_or(index); + + conn.send(&msg_term_output(NONCE + 1, id, index, max_bytes, 0)) + .await?; + let reply = await_output(&mut conn, NONCE + 1).await?; + check_status(reply.status, id)?; + + let truncated = reply.flags & OUTPUT_TRUNCATED != 0; + let evicted = reply.flags & OUTPUT_EVICTED != 0; + if json { + let mut value = match &record { + Some(record) => record_json(record), + None => serde_json::json!({ "index": index }), + }; + let object = value.as_object_mut().expect("record json is an object"); + object.insert("text".into(), reply.text.clone().into()); + object.insert("truncated".into(), truncated.into()); + object.insert("output_evicted".into(), evicted.into()); + object.insert( + "next_cursor".into(), + format_cursor(reply.next_seq, reply.next_col).into(), + ); + println!("{value}"); + } else { + print!("{}", reply.text); + if !reply.text.ends_with('\n') && !reply.text.is_empty() { + println!(); + } + if evicted { + eprintln!("blit: output start had scrolled out of the backlog"); + } + if truncated { + eprintln!( + "blit: truncated at {max_bytes} bytes; continue from cursor {}", + format_cursor(reply.next_seq, reply.next_col) + ); + } + } + + // With --wait the command's own status is the useful exit code; a plain + // fetch only reports whether the fetch worked. + Ok(match record.as_ref().and_then(|r| r.exit()) { + Some(code) if code >= 0 => code.min(255), + Some(code) => 128 + (-code).min(127), + // Timed out: the record came back still running. + None if record.as_ref().is_some_and(|r| r.running()) => 124, + None => 0, + }) +} + +/// `blit terminal history ID --since CURSOR` — everything appended since a +/// cursor, with the cursor to use next time. +pub async fn cmd_since( + transport: Transport, + id: u16, + cursor: Cursor, + max_bytes: u32, + json: bool, +) -> Result { + let mut conn = AgentConn::connect(transport).await?; + require_feature(&conn, id)?; + + let reply = match cursor { + Cursor::Now => { + let (seq, col) = probe_cursor(&mut conn, id, NONCE).await?; + OutputReply { + nonce: NONCE, + pty_id: id, + status: STATUS_OK, + flags: 0, + start_seq: seq, + start_col: col, + next_seq: seq, + next_col: col, + text: String::new(), + } + } + Cursor::At(seq, col) => { + conn.send(&msg_term_since(NONCE, id, seq, col, max_bytes, 0)) + .await?; + let reply = await_output(&mut conn, NONCE).await?; + check_status(reply.status, id)?; + reply + } + }; + + let next = format_cursor(reply.next_seq, reply.next_col); + if json { + println!( + "{}", + serde_json::json!({ + "pty": id, + "text": reply.text, + "cursor": format_cursor(reply.start_seq, reply.start_col), + "next_cursor": next, + "truncated": reply.flags & OUTPUT_TRUNCATED != 0, + "evicted": reply.flags & OUTPUT_EVICTED != 0, + "alt_screen": reply.flags & OUTPUT_ALT_SCREEN != 0, + }) + ); + } else { + print!("{}", reply.text); + if !reply.text.is_empty() && !reply.text.ends_with('\n') { + println!(); + } + // stdout stays exactly the terminal's bytes, so the cursor — which + // the caller needs for the next read — goes to stderr. + eprintln!("cursor: {next}"); + } + Ok(0) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn cursors_parse_in_every_spelling() { + assert!(matches!(parse_cursor("now").unwrap(), Cursor::Now)); + assert!(matches!(parse_cursor("end").unwrap(), Cursor::Now)); + assert!(matches!(parse_cursor("start").unwrap(), Cursor::At(0, 0))); + assert!(matches!(parse_cursor("42").unwrap(), Cursor::At(42, 0))); + assert!(matches!(parse_cursor("42:7").unwrap(), Cursor::At(42, 7))); + } + + #[test] + fn a_printed_cursor_parses_back() { + let text = format_cursor(1234, 56); + assert!(matches!(parse_cursor(&text).unwrap(), Cursor::At(1234, 56))); + } + + #[test] + fn nonsense_cursors_are_rejected() { + for text in ["", "-1", "abc", "1:2:3", "1:x", "1:70000"] { + assert!(parse_cursor(text).is_err(), "{text} must not parse"); + } + } + + #[test] + fn status_names_follow_the_flags() { + let running = CommandRecord { + flags: blit_remote::journal::RECORD_RUNNING, + ..CommandRecord::default() + }; + assert_eq!(record_status(&running), "running"); + let exited = CommandRecord { + flags: RECORD_HAS_EXIT, + exit_code: 3, + ..CommandRecord::default() + }; + assert_eq!(record_status(&exited), "exited"); + assert_eq!(exited.exit(), Some(3)); + let interrupted = CommandRecord { + flags: RECORD_INCOMPLETE, + ..CommandRecord::default() + }; + assert_eq!(record_status(&interrupted), "incomplete"); + assert_eq!(record_status(&CommandRecord::default()), "done"); + } + + #[test] + fn duration_needs_both_timestamps() { + let record = CommandRecord { + started_ms: 1000, + ended_ms: 1750, + ..CommandRecord::default() + }; + assert_eq!(duration_ms(&record), Some(750)); + assert_eq!( + duration_ms(&CommandRecord { + started_ms: 1000, + ..CommandRecord::default() + }), + None + ); + } + + #[test] + fn json_carries_what_an_agent_branches_on() { + let record = CommandRecord { + index: 4, + flags: RECORD_HAS_EXIT, + exit_code: 1, + start_seq: 100, + end_seq: 120, + started_ms: 10, + ended_ms: 30, + command: "false".into(), + }; + let value = record_json(&record); + assert_eq!(value["index"], 4); + assert_eq!(value["exit"], 1); + assert_eq!(value["running"], false); + assert_eq!(value["status"], "exited"); + assert_eq!(value["duration_ms"], 20); + assert_eq!(value["cursor"], "100:0"); + } +} diff --git a/crates/cli/src/learn.md b/crates/cli/src/learn.md index 9ffc3b54..edd10d3d 100644 --- a/crates/cli/src/learn.md +++ b/crates/cli/src/learn.md @@ -41,8 +41,32 @@ CR. Use `\x0a` if you need a literal LF byte. - `blit terminal show ID` — current viewport (what's on screen now) - `blit terminal history ID --from-end 0 --limit N` — last N lines from scrollback +- `blit terminal history ID --since CURSOR` — only what is new since `SEQ`, `SEQ:COL`, `now`, or `start` -Both accept `--cols`/`--rows` to resize before reading, and `--ansi` to preserve colors. +Always pass `--limit` or `--since`. Bare `history` is the whole scrollback. + +`--since` prints a cursor to feed back in. `--json` is the structured form +(text plus the next cursor). Default cap is 256 KiB; the reply says where to +continue from if it truncated. + +Both `show` and line-based `history` accept `--cols`/`--rows` to resize before +reading, and `--ansi` to preserve colors. + +## Commands in a live shell + +A shell that emits OSC 133 records each command (see +`docs/shell-integration.md`). Without that, the journal is empty. + +```bash +blit terminal send "$ID" "cargo test\n" +blit terminal output "$ID" --wait 600 # that command's output; exits with its status +blit terminal journal "$ID" # INDEX STATUS EXIT MS START_SEQ END_SEQ COMMAND +blit terminal output "$ID" 3 # command 3's output +blit terminal journal "$ID" --json +``` + +`--wait` blocks server-side until the command finishes (exit 124 on timeout). +`wait --pattern` matches only output produced after the wait began. ## Terminal lifecycle diff --git a/crates/cli/src/main.rs b/crates/cli/src/main.rs index b8d73c4a..8bf5a9d0 100644 --- a/crates/cli/src/main.rs +++ b/crates/cli/src/main.rs @@ -9,6 +9,7 @@ mod generate; mod git; mod grep; mod interactive; +mod journal; mod kv; mod lsp; mod relay; @@ -179,13 +180,58 @@ async fn async_main() { from_start, from_end, limit, + since, + max_bytes, + json, ansi, rows, cols, } => { + if let Some(since) = since { + let cursor = match journal::parse_cursor(&since) { + Ok(c) => c, + Err(e) => { + eprintln!("blit: {e}"); + std::process::exit(2); + } + }; + let max_bytes = max_bytes.unwrap_or(journal::OUTPUT_MAX_BYTES); + match journal::cmd_since(transport, id, cursor, max_bytes, json).await { + Ok(code) => std::process::exit(code), + Err(e) => { + eprintln!("blit: {e}"); + std::process::exit(1); + } + } + } let size = agent::capture_size(rows, cols); agent::cmd_history(transport, id, from_start, from_end, limit, ansi, size).await } + TerminalCommand::Journal { + id, + from, + limit, + json, + } => match journal::cmd_journal(transport, id, from, limit, json).await { + Ok(code) => std::process::exit(code), + Err(e) => { + eprintln!("blit: {e}"); + std::process::exit(1); + } + }, + TerminalCommand::Output { + id, + index, + wait, + max_bytes, + json, + } => match journal::cmd_output(transport, id, index, wait, max_bytes, json).await { + Ok(code) => std::process::exit(code), + Err(e) => { + eprintln!("blit: {e}"); + std::process::exit(1); + } + }, TerminalCommand::Send { id, text } => { let text = if text == "-" { use std::io::Read; diff --git a/crates/compositor/src/imp.rs b/crates/compositor/src/imp.rs index 254c4970..b4042cf4 100644 --- a/crates/compositor/src/imp.rs +++ b/crates/compositor/src/imp.rs @@ -661,6 +661,18 @@ pub enum CompositorEvent { surface_id: u16, app_id: String, }, + /// The stamped identity of a toplevel's application, sent once at creation + /// for surfaces that arrived on a per-app socket. + /// + /// A separate event rather than a field on `SurfaceCreated` because it + /// applies to a minority of surfaces and because `SurfaceAppId` already + /// established that shape for per-surface metadata. + SurfaceOrigin { + surface_id: u16, + sandbox_engine: String, + app_id: String, + instance_id: String, + }, SurfaceResized { surface_id: u16, width: u16, @@ -697,7 +709,40 @@ pub enum CompositorEvent { }, } +/// Who a Wayland connection belongs to. +/// +/// Stamped by whoever created the socket the client arrived on, never asserted +/// by the client itself — which is the whole point. `app_id` from +/// `xdg_toplevel.set_app_id` is a free-form string an application says about +/// itself, unverified and often wrong; `SO_PEERCRED` gives a pid that a +/// zygote-forking or re-execing application immediately invalidates, and a +/// passed connection fd means one socket need not mean one process at all. +/// +/// The fields mirror `wp_security_context_v1` so that protocol can be wired to +/// this later, letting a third-party sandbox engine stamp its own sockets. +/// Nothing here depends on that protocol existing. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct AppIdentity { + /// What created the socket, e.g. `blit` for the session supervisor. + pub sandbox_engine: String, + /// Stable across restarts of the same application. + pub app_id: String, + /// Distinguishes two concurrent runs of one application. + pub instance_id: String, +} + pub enum CompositorCommand { + /// Adopt an already-bound listening socket whose clients are known to + /// belong to `identity`. + /// + /// The caller binds the socket so the app can be spawned the instant this is + /// sent — there is no window in which the socket is named but not yet + /// listening. Ownership of the path stays with the caller, which unlinks it; + /// the compositor only accepts on the fd. + AddAppSocket { + fd: OwnedFd, + identity: AppIdentity, + }, KeyInput { surface_id: u16, keycode: u32, @@ -1478,6 +1523,56 @@ struct ClientState { cleanup_needed: Arc, } +/// Accept one connection and register it, optionally stamped with the identity +/// of the socket it arrived on. +/// +/// Shared by the session's own socket and every per-app socket so the two cannot +/// drift: a stamped client differs only in carrying an identity. +fn accept_client( + state: &mut Compositor, + client_stream: std::os::unix::net::UnixStream, + identity: Option>, + monitor_cancel: &std::os::unix::net::UnixStream, +) { + let watched_stream = client_stream.try_clone().ok(); + match state.display_handle.insert_client( + client_stream, + Arc::new(ClientState { + cleanup_needed: Arc::clone(&state.cleanup_needed), + }), + ) { + Ok(client) => { + // The peer's pid is what tells the X11 bridge apart + // from an ordinary app. + if let Ok(creds) = client.get_credentials(&state.display_handle) + && creds.pid > 0 + { + state.note_client_pid(client.id(), creds.pid as u32); + } + if let Some(identity) = identity { + state.client_identity.insert(client.id(), identity); + } + // Offer the screen before the client reads its + // registry: a toolkit that finds no output there + // never opens a window at all. + state.ensure_client_output(client.id()); + if let Some(watched_stream) = watched_stream { + monitor_client_disconnect( + watched_stream, + monitor_cancel, + state.display_handle.backend_handle(), + client.id(), + state.verbose, + ); + } + } + Err(e) if state.verbose => { + eprintln!("[compositor] insert_client error: {e}"); + } + Err(_) => {} + } +} + /// Stop dispatching a client's stale request backlog once its socket closes. /// /// wayland-backend drains every already-readable request before it reads EOF. @@ -2153,6 +2248,9 @@ struct Compositor { client_pids: FxHashMap, /// Clients resolved to belong to the bridge's process tree. xwayland_clients: FxHashSet, + /// Identity of clients that arrived on a stamped per-app socket. Absent for + /// the shared socket, where nothing can be said about who connected. + client_identity: FxHashMap>, seats: Vec, keyboards: Vec, pointers: Vec, @@ -2647,9 +2745,18 @@ impl Compositor { // so they go out ahead of the `enter` below, not at bind time. self.offer_selections_to_client(wl); let serial = self.next_serial(); + // A client's modifier state starts empty and only ever moves on a + // `modifiers` event, which `update_and_send_modifiers` sends solely as + // a side effect of a modifier key transition — and it sends it to + // whoever held focus at the time. So focus landing here while Ctrl is + // held leaves this client believing nothing is down, and it reads the + // next keystroke unmodified. State the seat-wide fact it has no other + // way to learn, the way the pointer's bind-time `enter` replay does. + let mods_serial = self.next_serial(); for kb in &self.keyboards { if same_client(kb, wl) { kb.enter(serial, wl, vec![]); + kb.modifiers(mods_serial, self.mods_depressed, 0, self.mods_locked, 0); } } for ti in &mut self.text_inputs { @@ -3268,6 +3375,15 @@ impl Compositor { Some(surf.wl_surface.client()?.id()) } + /// The stamped identity of the application a toplevel belongs to. + /// + /// `None` for anything on the shared socket, which is most windows: nothing + /// is known about who connected there, and guessing from a self-asserted + /// `app_id` would be worse than admitting it. + fn surface_identity(&self, surface_id: u16) -> Option<&Arc> { + self.client_identity.get(&self.surface_owner(surface_id)?) + } + /// Whether this client is the X11 bridge, whose screens are shared by /// every X window rather than owned one apiece. fn is_xwayland(&self, owner: &ClientId) -> bool { @@ -4664,6 +4780,8 @@ impl Compositor { .retain(|client, _| backend.get_client_data(client.clone()).is_ok()); self.xwayland_clients .retain(|client| backend.get_client_data(client.clone()).is_ok()); + self.client_identity + .retain(|client, _| backend.get_client_data(client.clone()).is_ok()); // Disabled globals remain bindable specifically so a client cannot // lose a race with `global_remove`. Once that client is dead there // can be no in-flight bind, and no other client was ever allowed to @@ -5055,6 +5173,19 @@ impl Compositor { fn handle_command(&mut self, cmd: CompositorCommand) { match cmd { + // Registering a socket needs the event loop's handle, which this + // method does not have, so the command loop intercepts it before + // dispatching here — same reason `Shutdown` is handled there. + // Reaching this arm would mean that interception was lost, and + // silently dropping the fd would strand an app with a socket + // nothing ever accepts on. + CompositorCommand::AddAppSocket { identity, .. } => { + eprintln!( + "[compositor] BUG: AddAppSocket for {} reached handle_command; \ + the command loop must intercept it", + identity.app_id + ); + } CompositorCommand::KeyInput { surface_id: _, keycode, @@ -7843,6 +7974,18 @@ impl Dispatch for Compositor { width: 0, height: 0, }); + // Sent right after creation so a subscriber never sees the + // surface without knowing whose it is. Identity is fixed for + // the life of the connection, so this is the only time it can + // change. + if let Some(identity) = state.surface_identity(surface_id) { + let _ = state.event_tx.send(CompositorEvent::SurfaceOrigin { + surface_id, + sandbox_engine: identity.sandbox_engine.clone(), + app_id: identity.app_id.clone(), + instance_id: identity.instance_id.clone(), + }); + } (state.event_notify)(); if state.verbose { eprintln!("[compositor] new_toplevel sid={surface_id}"); @@ -11646,6 +11789,7 @@ fn run_compositor( last_topless_frame_ms: FxHashMap::default(), pending_request_frames: FxHashMap::default(), frame_callback_toplevels: FxHashSet::default(), + client_identity: FxHashMap::default(), next_surface_id: 1, shm_pools: FxHashMap::default(), surface_meta: FxHashMap::default(), @@ -11767,40 +11911,9 @@ fn run_compositor( .insert_source(socket_source, move |_, socket, state| { let ls = unsafe { socket.get_mut() }; if let Some(client_stream) = ls.accept().ok().flatten() { - let watched_stream = client_stream.try_clone().ok(); - match state.display_handle.insert_client( - client_stream, - Arc::new(ClientState { - cleanup_needed: Arc::clone(&state.cleanup_needed), - }), - ) { - Ok(client) => { - // The peer's pid is what tells the X11 bridge apart - // from an ordinary app. - if let Ok(creds) = client.get_credentials(&state.display_handle) - && creds.pid > 0 - { - state.note_client_pid(client.id(), creds.pid as u32); - } - // Offer the screen before the client reads its - // registry: a toolkit that finds no output there - // never opens a window at all. - state.ensure_client_output(client.id()); - if let Some(watched_stream) = watched_stream { - monitor_client_disconnect( - watched_stream, - &monitor_cancel, - state.display_handle.backend_handle(), - client.id(), - state.verbose, - ); - } - } - Err(e) if state.verbose => { - eprintln!("[compositor] insert_client error: {e}"); - } - Err(_) => {} - } + // The shared socket says nothing about who connected: anything + // that inherited WAYLAND_DISPLAY can reach it. + accept_client(state, client_stream, None, &monitor_cancel); } Ok(PostAction::Continue) }) @@ -11837,6 +11950,53 @@ fn run_compositor( shutdown.store(true, Ordering::Relaxed); return; } + // Handled here rather than in `handle_command` because it needs + // the event loop's handle to register a new source, which a + // `&mut Compositor` method has no access to. + CompositorCommand::AddAppSocket { fd, identity } => { + let listener = std::os::unix::net::UnixListener::from(fd); + if let Err(e) = listener.set_nonblocking(true) { + eprintln!("[compositor] app socket set_nonblocking failed: {e}"); + continue; + } + let identity = Arc::new(identity); + let label = identity.app_id.clone(); + let Ok(cancel) = monitor_cancel_read.try_clone() else { + eprintln!("[compositor] app socket {label}: cancel fd clone failed"); + continue; + }; + let source = Generic::new(listener, Interest::READ, calloop::Mode::Level); + let inserted = handle.insert_source(source, move |_, listener, state| { + // Level-triggered, so one accept per readiness is + // enough; a second pending connection wakes us again. + match unsafe { listener.get_mut() }.accept() { + Ok((client_stream, _)) => { + accept_client( + state, + client_stream, + Some(Arc::clone(&identity)), + &cancel, + ); + } + Err(e) if e.kind() == std::io::ErrorKind::WouldBlock => {} + Err(e) if state.verbose => { + eprintln!("[compositor] app socket accept error: {e}"); + } + Err(_) => {} + } + Ok(PostAction::Continue) + }); + match inserted { + Ok(_token) => { + if verbose { + eprintln!("[compositor] app socket registered for {label}"); + } + } + Err(e) => { + eprintln!("[compositor] app socket {label} not registered: {e}"); + } + } + } other => compositor.handle_command(other), } } diff --git a/crates/compositor/src/lib.rs b/crates/compositor/src/lib.rs index e9fa2c4b..d3ab5edc 100644 --- a/crates/compositor/src/lib.rs +++ b/crates/compositor/src/lib.rs @@ -193,6 +193,14 @@ mod stub { surface_id: u16, app_id: String, }, + /// The stamped identity of a toplevel's application, sent once at + /// creation for surfaces that arrived on a per-app socket. + SurfaceOrigin { + surface_id: u16, + sandbox_engine: String, + app_id: String, + instance_id: String, + }, /// The client asked for one of its toplevels to be activated /// (xdg_activation_v1) — e.g. an Electron app reacting to a /// notification click. Pane focus belongs to the frontend, so the @@ -235,6 +243,28 @@ mod stub { }, } + /// Who a Wayland connection belongs to. + /// + /// Stamped by whoever created the socket the client arrived on, never + /// asserted by the client itself — which is the whole point. `app_id` from + /// `xdg_toplevel.set_app_id` is a free-form string an application says about + /// itself, unverified and often wrong; `SO_PEERCRED` gives a pid that a + /// zygote-forking or re-execing application immediately invalidates, and a + /// passed connection fd means one socket need not mean one process at all. + /// + /// The fields mirror `wp_security_context_v1` so that protocol can be wired + /// to this later, letting a third-party sandbox engine stamp its own + /// sockets. Nothing here depends on that protocol existing. + #[derive(Debug, Clone, PartialEq, Eq)] + pub struct AppIdentity { + /// What created the socket, e.g. `blit` for the session supervisor. + pub sandbox_engine: String, + /// Stable across restarts of the same application. + pub app_id: String, + /// Distinguishes two concurrent runs of one application. + pub instance_id: String, + } + pub enum CompositorCommand { KeyInput { surface_id: u16, @@ -410,6 +440,17 @@ mod stub { SetXwaylandPid { pid: u32, }, + /// Adopt an already-bound listening socket whose clients are known to + /// belong to `identity`. + /// + /// The caller binds the socket so the app can be spawned the instant + /// this is sent — there is no window in which the socket is named but + /// not yet listening. Ownership of the path stays with the caller, + /// which unlinks it; the compositor only accepts on the fd. + AddAppSocket { + fd: OwnedFd, + identity: AppIdentity, + }, /// Update the advertised output refresh rate (millihertz). SetRefreshRate { mhz: u32, diff --git a/crates/compositor/tests/keyboard_modifiers.rs b/crates/compositor/tests/keyboard_modifiers.rs new file mode 100644 index 00000000..e7c02c5b --- /dev/null +++ b/crates/compositor/tests/keyboard_modifiers.rs @@ -0,0 +1,364 @@ +//! A surface taking keyboard focus has to be told which modifiers are held. +//! +//! Modifier state is seat-wide but delivery is not: `wl_keyboard.modifiers` +//! goes to whoever holds focus when a modifier key moves, and a client's own +//! state starts empty and moves on nothing else. Ctrl held down while another +//! pane had focus is therefore a fact the newly focused client cannot derive +//! from anything it receives -- `enter` carries no modifiers of its own -- and +//! it reads the next keystroke unmodified. Ctrl+K becomes a bare K. +//! +//! The keymap here is the compositor's own US-QWERTY one, so these tests speak +//! in its modifier bits rather than in xkbcommon's. + +#![cfg(target_os = "linux")] + +use std::os::unix::net::UnixStream; +use std::sync::Arc; +use std::time::Duration; + +use wayland_client::protocol::{wl_compositor, wl_keyboard, wl_registry, wl_seat, wl_surface}; +use wayland_client::{Connection, Dispatch, EventQueue, QueueHandle, delegate_noop}; +use wayland_protocols::xdg::shell::client::{xdg_surface, xdg_toplevel, xdg_wm_base}; + +use blit_compositor::{ + CompositorCommand, CompositorEvent, CompositorHandle, spawn_compositor_without_renderer, +}; + +/// Modifier bits, as `data/us-qwerty.xkb` numbers them. +const MOD_SHIFT: u32 = 1 << 0; +const MOD_LOCK: u32 = 1 << 1; +const MOD_CONTROL: u32 = 1 << 2; + +const KEY_LEFTCTRL: u32 = 29; +const KEY_LEFTSHIFT: u32 = 42; +const KEY_CAPSLOCK: u32 = 58; +const KEY_K: u32 = 37; + +/// The keyboard events these tests care about, in arrival order -- the order is +/// half the property: a `modifiers` that trails the key it qualifies is a +/// modifier the app applied to the wrong keystroke. +#[derive(Debug, Clone, PartialEq, Eq)] +enum Ev { + Enter, + Leave, + /// modifiers(depressed, locked) + Mods(u32, u32), + Key(u32, bool), +} + +#[derive(Default)] +struct App { + compositor: Option, + wm_base: Option, + seat: Option, + log: Vec, +} + +impl Dispatch for App { + fn event( + state: &mut Self, + registry: &wl_registry::WlRegistry, + event: wl_registry::Event, + _: &(), + _: &Connection, + qh: &QueueHandle, + ) { + let wl_registry::Event::Global { + name, interface, .. + } = event + else { + return; + }; + match interface.as_str() { + "wl_compositor" => { + state.compositor = + Some(registry.bind::(name, 4, qh, ())); + } + "xdg_wm_base" => { + state.wm_base = + Some(registry.bind::(name, 1, qh, ())); + } + "wl_seat" => { + state.seat = Some(registry.bind::(name, 7, qh, ())); + } + _ => {} + } + } +} + +impl Dispatch for App { + fn event( + _: &mut Self, + wm_base: &xdg_wm_base::XdgWmBase, + event: xdg_wm_base::Event, + _: &(), + _: &Connection, + _: &QueueHandle, + ) { + if let xdg_wm_base::Event::Ping { serial } = event { + wm_base.pong(serial); + } + } +} + +impl Dispatch for App { + fn event( + _: &mut Self, + xdg_surface: &xdg_surface::XdgSurface, + event: xdg_surface::Event, + _: &(), + _: &Connection, + _: &QueueHandle, + ) { + if let xdg_surface::Event::Configure { serial } = event { + xdg_surface.ack_configure(serial); + } + } +} + +impl Dispatch for App { + fn event( + state: &mut Self, + _: &wl_keyboard::WlKeyboard, + event: wl_keyboard::Event, + _: &(), + _: &Connection, + _: &QueueHandle, + ) { + match event { + wl_keyboard::Event::Enter { .. } => state.log.push(Ev::Enter), + wl_keyboard::Event::Leave { .. } => state.log.push(Ev::Leave), + wl_keyboard::Event::Modifiers { + mods_depressed, + mods_locked, + .. + } => state.log.push(Ev::Mods(mods_depressed, mods_locked)), + wl_keyboard::Event::Key { key, state: s, .. } => { + let pressed = matches!(s.into_result(), Ok(wl_keyboard::KeyState::Pressed)); + state.log.push(Ev::Key(key, pressed)); + } + _ => {} + } + } +} + +delegate_noop!(App: ignore wl_compositor::WlCompositor); +delegate_noop!(App: ignore wl_surface::WlSurface); +delegate_noop!(App: ignore xdg_toplevel::XdgToplevel); +delegate_noop!(App: ignore wl_seat::WlSeat); + +struct Fixture { + app: App, + queue: EventQueue, + conn: Connection, + surface_id: u16, + _surface: wl_surface::WlSurface, + _toplevel: xdg_toplevel::XdgToplevel, + _xdg_surface: xdg_surface::XdgSurface, + handle: Option, +} + +impl Drop for Fixture { + fn drop(&mut self) { + if let Some(handle) = self.handle.take() { + handle.stop(); + } + } +} + +impl Fixture { + /// A mapped toplevel with a `wl_keyboard`, wound back to unfocused: every + /// test here is about what focus itself delivers, and mapping a toplevel + /// hands it focus already. + fn new() -> Self { + let handle = spawn_compositor_without_renderer(false, Arc::new(|| {})); + let stream = + UnixStream::connect(&handle.socket_name).expect("connect to compositor socket"); + let conn = Connection::from_socket(stream).expect("wayland connection"); + let mut queue = conn.new_event_queue(); + let qh = queue.handle(); + conn.display().get_registry(&qh, ()); + + let mut app = App::default(); + queue.roundtrip(&mut app).expect("registry roundtrip"); + let compositor = app.compositor.clone().expect("wl_compositor advertised"); + let wm_base = app.wm_base.clone().expect("xdg_wm_base advertised"); + let seat = app.seat.clone().expect("wl_seat advertised"); + + let _keyboard = seat.get_keyboard(&qh, ()); + let surface = compositor.create_surface(&qh, ()); + let xdg_surface = wm_base.get_xdg_surface(&surface, &qh, ()); + let toplevel = xdg_surface.get_toplevel(&qh, ()); + surface.commit(); + queue.roundtrip(&mut app).expect("map roundtrip"); + + let mut fx = Self { + app, + queue, + conn, + surface_id: 0, + _surface: surface, + _toplevel: toplevel, + _xdg_surface: xdg_surface, + handle: Some(handle), + }; + fx.surface_id = fx.await_surface_id(); + fx.settle(); + fx.focus(0); + fx.take_log(); + fx + } + + fn command(&self, cmd: CompositorCommand) { + let handle = self.handle.as_ref().expect("compositor running"); + handle.command_tx.send(cmd).expect("send command"); + handle.wake(); + } + + /// Flush our requests, let the compositor's own thread act, read back. + fn settle(&mut self) { + self.conn.flush().expect("flush"); + std::thread::sleep(Duration::from_millis(50)); + self.queue.roundtrip(&mut self.app).expect("roundtrip"); + } + + fn focus(&mut self, surface_id: u16) { + self.command(CompositorCommand::SurfaceFocus { surface_id }); + self.settle(); + } + + /// Type a key the way the browser's `C2S_SURFACE_INPUT` does. + fn key(&mut self, keycode: u32, pressed: bool) { + let surface_id = self.surface_id; + self.command(CompositorCommand::KeyInput { + surface_id, + keycode, + pressed, + time_ms: 0, + }); + self.settle(); + } + + fn tap(&mut self, keycode: u32) { + self.key(keycode, true); + self.key(keycode, false); + } + + fn take_log(&mut self) -> Vec { + std::mem::take(&mut self.app.log) + } + + fn await_surface_id(&mut self) -> u16 { + let handle = self.handle.as_ref().expect("compositor running"); + let deadline = std::time::Instant::now() + Duration::from_secs(5); + while std::time::Instant::now() < deadline { + if let Ok(CompositorEvent::SurfaceCreated { surface_id, .. }) = + handle.event_rx.recv_timeout(Duration::from_millis(200)) + { + return surface_id; + } + } + panic!("compositor never announced the surface"); + } +} + +#[test] +fn focus_states_the_modifiers_already_held() { + // The bug: Ctrl goes down while this surface has no focus, so its own + // client is never in the set `update_and_send_modifiers` writes to. Focus + // then arrives and the client has been told nothing at all. + let mut fx = Fixture::new(); + fx.key(KEY_LEFTCTRL, true); + assert_eq!( + fx.take_log(), + vec![], + "an unfocused client should hear nothing about the press" + ); + + fx.focus(fx.surface_id); + + assert_eq!( + fx.take_log(), + vec![Ev::Enter, Ev::Mods(MOD_CONTROL, 0)], + "focus should restate the held modifier, and after the enter it qualifies" + ); +} + +#[test] +fn a_chord_typed_right_after_focus_is_modified() { + // What the user actually does: hold Ctrl somewhere else, click into the + // app, press K. The app has to see the K as Ctrl+K. + let mut fx = Fixture::new(); + fx.key(KEY_LEFTCTRL, true); + + fx.focus(fx.surface_id); + fx.tap(KEY_K); + + assert_eq!( + fx.take_log(), + vec![ + Ev::Enter, + Ev::Mods(MOD_CONTROL, 0), + Ev::Key(KEY_K, true), + Ev::Key(KEY_K, false), + ], + "Ctrl has to be stated ahead of the K it qualifies, or the app types a bare k" + ); +} + +#[test] +fn focus_states_an_empty_modifier_set_too() { + // Not just a nicety: without it the app keeps whatever it was last told, + // and a client that had focus while Ctrl was held, lost it, and got it + // back would still be holding Ctrl after the user let go. + let mut fx = Fixture::new(); + fx.focus(fx.surface_id); + assert_eq!(fx.take_log(), vec![Ev::Enter, Ev::Mods(0, 0)]); + + fx.key(KEY_LEFTCTRL, true); + fx.focus(0); + fx.key(KEY_LEFTCTRL, false); + fx.take_log(); + + fx.focus(fx.surface_id); + + assert_eq!( + fx.take_log(), + vec![Ev::Enter, Ev::Mods(0, 0)], + "the release happened elsewhere, so focus has to say Ctrl is up" + ); +} + +#[test] +fn focus_carries_the_capslock_latch() { + // A locked modifier survives focus moving away and back, so it is the one + // piece of state that is definitely still true when the client returns. + let mut fx = Fixture::new(); + fx.focus(fx.surface_id); + fx.tap(KEY_CAPSLOCK); + fx.focus(0); + fx.take_log(); + + fx.focus(fx.surface_id); + + assert_eq!( + fx.take_log(), + vec![Ev::Enter, Ev::Mods(0, MOD_LOCK)], + "CapsLock is latched on, and only a modifiers event can say so" + ); +} + +#[test] +fn several_held_modifiers_arrive_together() { + let mut fx = Fixture::new(); + fx.key(KEY_LEFTCTRL, true); + fx.key(KEY_LEFTSHIFT, true); + + fx.focus(fx.surface_id); + + assert_eq!( + fx.take_log(), + vec![Ev::Enter, Ev::Mods(MOD_CONTROL | MOD_SHIFT, 0)], + "one modifiers event carries the whole mask, not one per key" + ); +} diff --git a/crates/git/src/lib.rs b/crates/git/src/lib.rs index 442ad1c3..7a0eceb7 100644 --- a/crates/git/src/lib.rs +++ b/crates/git/src/lib.rs @@ -74,6 +74,11 @@ pub struct Budgets { /// real UI — the cap only stops a client exhausting memory with distinct /// ids. pub max_log_subs: usize, + /// Worktrees one `GIT_WORKTREES` response describes before it truncates + /// with a `CURSOR`. Its own budget, well below `entries_max`, because + /// each record costs a repository open to resolve that worktree's HEAD + /// — this bounds opens, not bytes. + pub worktrees_max: usize, } impl Default for Budgets { @@ -89,6 +94,7 @@ impl Default for Budgets { rename_limit: env_u64("BLIT_GIT_RENAME_LIMIT", 1_000) as usize, blame_lines_max: env_u64("BLIT_GIT_BLAME_LINES_MAX", 50_000).max(1) as u32, max_log_subs: env_u64("BLIT_GIT_MAX_LOG_SUBS", 64) as usize, + worktrees_max: env_u64("BLIT_GIT_WORKTREES_MAX", 256).max(1) as usize, } } } @@ -304,7 +310,7 @@ pub fn open(path: &str) -> Result<(RepoHandle, RepoInfo), (u8, String)> { Ok((handle, info)) } -fn canonical(path: &Path) -> std::path::PathBuf { +pub(crate) fn canonical(path: &Path) -> std::path::PathBuf { path.canonicalize().unwrap_or_else(|_| path.to_path_buf()) } diff --git a/crates/git/src/reads.rs b/crates/git/src/reads.rs index 31818f81..49ac0f9f 100644 --- a/crates/git/src/reads.rs +++ b/crates/git/src/reads.rs @@ -9,10 +9,13 @@ use blit_remote::git::{ GIT_FETCH_REF_TAG_UPDATE, GIT_FOUND_BARE, GIT_FOUND_LINKED, GIT_OID_NONE, GIT_REFLOG_OLDEST_FIRST, GIT_REFLOG_TRUNCATED, GIT_STATUS_CANCELLED, GIT_STATUS_INVALID, GIT_STATUS_NOT_FOUND, GIT_STATUS_OK, GIT_STATUS_OTHER, GIT_STATUS_PERMISSION, - GIT_STATUS_WRONG_TYPE, GitBlameRecord, GitBlameRequest, GitDiscoverRecord, GitDiscoverRequest, - GitFetchRecord, GitFetchRequest, GitReflogRecord, GitReflogRequest, append_git_blame_record, - append_git_discover_record, append_git_fetch_record, append_git_reflog_record, - msg_git_blame_resp, msg_git_discover_resp, msg_git_fetch_resp, msg_git_reflog_resp, + GIT_STATUS_WRONG_TYPE, GIT_WORKTREE_BARE, GIT_WORKTREE_CURRENT, GIT_WORKTREE_DETACHED, + GIT_WORKTREE_LOCKED, GIT_WORKTREE_MAIN, GIT_WORKTREE_PRUNABLE, GIT_WORKTREES_TRUNCATED, + GitBlameRecord, GitBlameRequest, GitDiscoverRecord, GitDiscoverRequest, GitFetchRecord, + GitFetchRequest, GitReflogRecord, GitReflogRequest, GitWorktreeRecord, GitWorktreesRequest, + append_git_blame_record, append_git_discover_record, append_git_fetch_record, + append_git_reflog_record, append_git_worktree_record, msg_git_blame_resp, + msg_git_discover_resp, msg_git_fetch_resp, msg_git_reflog_resp, msg_git_worktrees_resp, }; use crate::{Cancel, RepoHandle, oid_bytes}; @@ -348,6 +351,188 @@ impl RepoHandle { } msg_git_reflog_resp(nonce, GIT_STATUS_OK, flags, &records) } + + /// `GIT_WORKTREES`: the main worktree plus every linked one. + /// + /// gix lists only the *linked* worktrees, and from whichever worktree + /// the repo was opened through, so the main one is resolved separately + /// and always reported first — otherwise a client opened inside a + /// linked worktree could not name the checkout it forked from, which is + /// the one it most wants to get back to. + /// + /// Each record costs opening that worktree's gitdir, because a + /// worktree's HEAD is per-worktree state that the shared ref store + /// cannot answer for. That is what `worktrees_max` bounds. + pub fn worktrees(&self, req: &GitWorktreesRequest, cancel: &Cancel) -> Vec { + let nonce = req.nonce; + let fail = |status: u8| msg_git_worktrees_resp(nonce, status, 0, &[]); + if req.flags != 0 { + return fail(GIT_STATUS_INVALID); + } + let repo = self.local(); + // The worktree this repo_id was opened at, so `CURRENT` is decided + // by identity rather than by the client comparing paths it may have + // canonicalized differently than the server did. + let current = repo.workdir().map(crate::canonical); + // Already the main repo when our gitdir *is* the common dir; a + // linked worktree's gitdir is `/worktrees/`. Checking + // saves reopening the repository we are holding. + let main_owned = if repo.git_dir() == repo.common_dir() { + None + } else { + // `ok()`: the common dir being unreadable means the repository is + // coming apart under us. Every linked worktree still resolves, so + // report those rather than failing the whole enumeration. + repo.main_repo().ok() + }; + let main = main_owned.as_ref().unwrap_or(&repo); + + let skip = usize::try_from(req.after_pos).unwrap_or(usize::MAX); + let limit = self.budgets.worktrees_max; + let mut records = Vec::new(); + let mut emitted = 0usize; + let mut seen = 0usize; + let mut truncated = false; + + // `main_owned.is_none()` is not enough for CURRENT: the repo may + // have been opened at the main worktree through a path that made + // `main_repo()` run anyway, so compare the resolved workdirs. + let mut push = |flags: u8, oid, path: &str, branch: &str, lock: &str| { + append_git_worktree_record( + &mut records, + &GitWorktreeRecord::Tree { + flags, + oid, + path, + branch, + lock_reason: lock, + }, + ); + }; + + // ── the main worktree ──────────────────────────────────────────── + seen += 1; + if seen > skip { + let mut flags = GIT_WORKTREE_MAIN; + let path = match main.workdir() { + Some(dir) => { + let dir = crate::canonical(dir); + if current.as_deref() == Some(dir.as_path()) { + flags |= GIT_WORKTREE_CURRENT; + } + blit_fssync::escape_path(&dir) + } + None => { + // Bare: no checkout to navigate to, but naming it is + // how a client explains where the linked worktrees hang + // off. + flags |= GIT_WORKTREE_BARE; + String::new() + } + }; + let (oid, branch, detached) = worktree_head(main); + if detached { + flags |= GIT_WORKTREE_DETACHED; + } + push(flags, oid, &path, &branch, ""); + emitted += 1; + } + + // ── the linked worktrees, in gix's gitdir order ────────────────── + for proxy in repo.worktrees().unwrap_or_default() { + if cancel.is_cancelled() { + return fail(GIT_STATUS_CANCELLED); + } + seen += 1; + if seen <= skip { + continue; + } + if emitted >= limit { + truncated = true; + break; + } + let mut flags = 0u8; + // Read the administrative state before `into_repo…` consumes + // the proxy. + let lock = if proxy.is_locked() { + flags |= GIT_WORKTREE_LOCKED; + proxy + .lock_reason() + .map(|r| crate::escape_bstr(r.as_ref())) + .unwrap_or_default() + } else { + String::new() + }; + let base = proxy.base().ok(); + // `git worktree prune` drops an entry whose checkout is gone. + // That is the case a client has to be told about — a row it + // cannot navigate to — so it is reported rather than hidden. + // Narrower than git's own prunability test, which also covers + // an unparsable `gitdir` file; those fail `base()` and land + // here too. + if !base.as_deref().is_some_and(|p| p.is_dir()) { + flags |= GIT_WORKTREE_PRUNABLE; + } + let base = base.map(|p| crate::canonical(&p)); + if base.is_some() && base.as_deref() == current.as_deref() { + flags |= GIT_WORKTREE_CURRENT; + } + let path = base + .map(|p| blit_fssync::escape_path(&p)) + .unwrap_or_default(); + let (oid, branch, detached) = + match proxy.into_repo_with_possibly_inaccessible_worktree() { + Ok(wt) => worktree_head(&wt), + // A worktree whose gitdir will not open still exists as an + // entry; report it with an unknown HEAD rather than + // dropping it from the list. + Err(_) => (GIT_OID_NONE, String::new(), false), + }; + if detached { + flags |= GIT_WORKTREE_DETACHED; + } + push(flags, oid, &path, &branch, &lock); + emitted += 1; + } + + let mut flags = 0u8; + if truncated { + flags |= GIT_WORKTREES_TRUNCATED; + append_git_worktree_record( + &mut records, + &GitWorktreeRecord::Cursor { + after: "", + pos: (skip + emitted) as u64, + }, + ); + } + msg_git_worktrees_resp(nonce, GIT_STATUS_OK, flags, &records) + } +} + +/// `(oid, branch, detached)` for one worktree's HEAD, in the shape the +/// `TREE` record wants: an escaped full ref name, empty when detached, and +/// a zero oid when unborn or unreadable. +fn worktree_head(repo: &gix::Repository) -> (blit_remote::git::GitOid, String, bool) { + let Ok(head) = repo.head() else { + return (GIT_OID_NONE, String::new(), false); + }; + match head.kind { + gix::head::Kind::Symbolic(reference) => ( + repo.head_id() + .map(|id| oid_bytes(id.as_ref())) + .unwrap_or(GIT_OID_NONE), + crate::escape_bstr(reference.name.as_bstr()), + false, + ), + gix::head::Kind::Detached { target, .. } => { + (oid_bytes(target.as_ref()), String::new(), true) + } + // Unborn is not detached: the branch is named and simply has no + // commit yet, which is exactly what a fresh `git worktree add -b` + // looks like before its first commit. + gix::head::Kind::Unborn(name) => (GIT_OID_NONE, crate::escape_bstr(name.as_bstr()), false), + } } /// Whether a fetch can be attempted at all: the operator switch, and a diff --git a/crates/git/src/state.rs b/crates/git/src/state.rs index 81f0a15e..90e68eab 100644 --- a/crates/git/src/state.rs +++ b/crates/git/src/state.rs @@ -419,6 +419,12 @@ struct Arms { /// [`Engine::ignore_watch_dirs`] excludes anything under them — so /// these arm once and are only re-armed when the configured path moves. ignore_paths: Vec, + /// `/worktrees` while it is armed, `None` while it does not + /// exist. Its own slot rather than a `gitdir_paths` entry because it + /// has to be re-checked every pass: the directory is created by the + /// *first* `git worktree add`, long after `arm_gitdir` has run once and + /// latched, and without a re-arm every later add would go unseen. + worktrees_path: Option, /// The worktree watch is up (a status subscriber exists). worktree: bool, /// Worktree directories armed one at a time (`NonRecursive`), the root @@ -861,6 +867,7 @@ impl Engine { watcher, gitdir_paths: Vec::new(), ignore_paths: Vec::new(), + worktrees_path: None, worktree: false, worktree_dirs: BTreeSet::new(), worktree_pruned: false, @@ -1072,6 +1079,46 @@ impl Engine { } self.publish_worktree_watches(); self.sync_ignore_watch(); + self.sync_worktrees_watch(); + } + + /// Arm (or drop) the watch on `/worktrees`, the directory + /// behind `WORKTREE_GEN`. + /// + /// Recursive, because the events that matter happen one level down: a + /// `git worktree move` rewrites `worktrees//gitdir` in place and a + /// lock creates `worktrees//locked`, neither of which touches the + /// mtime of the directory being watched. Idempotent and re-checked + /// every pass, so the directory appearing (first `worktree add`) or + /// vanishing (`worktree prune` of the last one) is picked up — the + /// first add is seen by the non-recursive watch on `common` regardless, + /// which is what brings us back here to arm for the second. + fn sync_worktrees_watch(&mut self) { + use notify::Watcher as _; + let want = self.common.join("worktrees"); + let want = want.is_dir().then_some(want); + let Some(arms) = &mut self.watch else { + return; + }; + if arms.worktrees_path == want { + return; + } + let Arms { + watcher, + worktrees_path, + .. + } = arms; + let mut paths = watcher.paths_mut(); + if let Some(old) = worktrees_path.take() { + let _ = paths.remove(&old); + } + if let Some(dir) = &want + && paths.add(dir, notify::RecursiveMode::Recursive).is_ok() + { + *worktrees_path = Some(dir.clone()); + } + let _ = paths.commit(); + arms.stream_rebuilt = true; } /// Reconcile the per-directory worktree watch set with the tree and @@ -1637,6 +1684,7 @@ impl Engine { op_record(repo, &mut base); special_ref_records(repo, &mut base); stash_records(repo, entries_max, &mut base); + worktree_gen_record(repo, &mut base); let remotes = demand.remotes.then(|| { let mut records = Vec::new(); remote_records(repo, &mut records); @@ -2457,6 +2505,74 @@ fn stash_records(repo: &gix::Repository, entries_max: usize, records: &mut Vec) { + // The main worktree is always one of them, and gix's `worktrees()` + // counts only the linked ones. A bare repository has no main worktree, + // so it contributes nothing. + let mut count: u32 = u32::from(!repo.is_bare()); + let mut digest: u64 = 0; + if let Ok(entries) = std::fs::read_dir(repo.common_dir().join("worktrees")) { + for entry in entries.flatten() { + let dir = entry.path(); + let gitdir = dir.join("gitdir"); + // Same test `worktrees()` uses to decide an entry is a + // worktree at all, so the count cannot disagree with the list. + if !gitdir.is_file() { + continue; + } + count = count.saturating_add(1); + let mut h = Fnv::new(); + h.write(blit_fssync::escape_path(&dir).as_bytes()); + if let Ok(mtime) = gitdir.metadata().and_then(|m| m.modified()) + && let Ok(since) = mtime.duration_since(std::time::UNIX_EPOCH) + { + h.write(&since.as_nanos().to_le_bytes()); + } + h.write(&[u8::from(dir.join("locked").is_file())]); + digest ^= h.finish(); + } + } + append_git_state_record(records, &GitStateRecord::WorktreeGen { count, digest }); +} + +/// FNV-1a, so the digest is reproducible across processes and releases — +/// `DefaultHasher`'s algorithm is explicitly not. +struct Fnv(u64); + +impl Fnv { + fn new() -> Self { + Fnv(0xcbf2_9ce4_8422_2325) + } + + fn write(&mut self, bytes: &[u8]) { + for &b in bytes { + self.0 ^= u64::from(b); + self.0 = self.0.wrapping_mul(0x100_0000_01b3); + } + } + + fn finish(&self) -> u64 { + self.0 + } +} + /// Every oid in a gitdir-root file, one per line — `MERGE_HEAD` holds /// several during an octopus merge. Empty when the file is absent or has /// no parseable hash. diff --git a/crates/git/tests/engine.rs b/crates/git/tests/engine.rs index 5679dbb9..6a56862a 100644 --- a/crates/git/tests/engine.rs +++ b/crates/git/tests/engine.rs @@ -5057,3 +5057,309 @@ fn submodule_reads_its_checked_out_head() { vec![("deps/mod".to_string(), b' ', b'M')] ); } + +// --------------------------------------------------------------------------- +// GIT_WORKTREES +// --------------------------------------------------------------------------- + +/// One decoded `WORKTREE` record, owned so it outlives the response buffer. +#[derive(Clone, Debug, PartialEq, Eq)] +struct Wt { + flags: u8, + oid: GitOid, + path: String, + branch: String, + lock: String, +} + +/// Decode a `GIT_WORKTREES` response, asserting the reply is well formed. +fn worktrees_of(handle: &RepoHandle, after_pos: u64) -> (u8, Vec) { + let resp = handle.worktrees( + &GitWorktreesRequest { + nonce: 7, + repo_id: 0, + flags: 0, + after_pos, + }, + &Cancel::default(), + ); + let (nonce, status, flags, records) = parse_git_worktrees_resp(&resp).unwrap(); + assert_eq!((nonce, status), (7, GIT_STATUS_OK)); + let trees = git_worktree_records(&records) + .filter_map(|r| match r { + GitWorktreeRecord::Tree { + flags, + oid, + path, + branch, + lock_reason, + } => Some(Wt { + flags, + oid, + path: path.to_string(), + branch: branch.to_string(), + lock: lock_reason.to_string(), + }), + GitWorktreeRecord::Cursor { .. } => None, + }) + .collect(); + (flags, trees) +} + +/// A fixture with a main worktree and three linked ones: on a new branch, +/// detached, and locked with a reason. +fn worktree_fixture() -> (PathBuf, PathBuf) { + let dir = temp_dir(); + git(&dir, &["init", "-b", "main"]); + std::fs::write(dir.join("a.txt"), "one\n").unwrap(); + git(&dir, &["add", "."]); + git(&dir, &["commit", "-m", "first"]); + let trees = temp_dir(); + let feature = trees.join("feature"); + git( + &dir, + &[ + "worktree", + "add", + "-b", + "feature", + feature.to_str().unwrap(), + ], + ); + let detached = trees.join("detached"); + git( + &dir, + &["worktree", "add", "--detach", detached.to_str().unwrap()], + ); + let pinned = trees.join("pinned"); + git( + &dir, + &["worktree", "add", "-b", "pinned", pinned.to_str().unwrap()], + ); + git( + &dir, + &[ + "worktree", + "lock", + "--reason", + "on usb", + pinned.to_str().unwrap(), + ], + ); + (dir, trees) +} + +/// The main worktree is reported first and exactly once, and every linked +/// worktree lands with the branch, detachment and lock state git itself +/// reports. +#[test] +fn worktrees_lists_main_and_linked() { + let (dir, trees) = worktree_fixture(); + let (handle, _info) = open(dir.to_str().unwrap()).unwrap(); + let (flags, got) = worktrees_of(&handle, 0); + assert_eq!(flags, 0, "four worktrees is nowhere near the budget"); + + // Oracle: git's own count. + let listed = git_out(&dir, &["worktree", "list", "--porcelain"]); + let expected = listed + .split("\n\n") + .filter(|s| !s.trim().is_empty()) + .count(); + assert_eq!(got.len(), expected, "same count as `git worktree list`"); + assert_eq!(got.len(), 4); + + // Main first, and it is the one we opened. + let main = &got[0]; + let (mflags, mpath, mbranch) = (main.flags, &main.path, &main.branch); + assert_ne!(mflags & GIT_WORKTREE_MAIN, 0, "main reported first"); + assert_ne!( + mflags & GIT_WORKTREE_CURRENT, + 0, + "opened at the main worktree" + ); + assert_eq!(mpath, dir.to_str().unwrap()); + assert_eq!(mbranch, "refs/heads/main"); + assert_eq!( + got.iter() + .filter(|t| t.flags & GIT_WORKTREE_MAIN != 0) + .count(), + 1, + "exactly one main" + ); + + let by_path = |p: &Path| { + got.iter() + .find(|t| t.path == p.to_str().unwrap()) + .unwrap_or_else(|| panic!("no record for {}", p.display())) + .clone() + }; + let feature = by_path(&trees.join("feature")); + let (fflags, foid, fbranch) = (feature.flags, feature.oid, &feature.branch); + assert_eq!(fflags, 0, "a plain linked worktree carries no flags"); + assert_eq!(fbranch, "refs/heads/feature"); + assert_eq!(foid, rev(&dir, "HEAD"), "its HEAD commit"); + + let det = by_path(&trees.join("detached")); + let (dflags, doid, dbranch) = (det.flags, det.oid, &det.branch); + assert_ne!(dflags & GIT_WORKTREE_DETACHED, 0, "DETACHED"); + assert_eq!(dbranch, "", "a detached HEAD names no branch"); + assert_eq!(doid, rev(&dir, "HEAD"), "detached still resolves an oid"); + + let pin = by_path(&trees.join("pinned")); + let (pflags, plock) = (pin.flags, &pin.lock); + assert_ne!(pflags & GIT_WORKTREE_LOCKED, 0, "LOCKED"); + assert_eq!(plock, "on usb", "the reason git recorded"); +} + +/// Opened *through a linked worktree* — the case that motivates the +/// request. gix lists only linked worktrees relative to wherever it was +/// opened, so without resolving the main repo separately a client inside a +/// linked worktree could not name the checkout it forked from. +#[test] +fn worktrees_from_a_linked_worktree_still_names_main() { + let (dir, trees) = worktree_fixture(); + let feature = trees.join("feature"); + let (handle, info) = open(feature.to_str().unwrap()).unwrap(); + assert_ne!(info.flags & GIT_REPO_LINKED, 0, "opened a linked worktree"); + let (_flags, got) = worktrees_of(&handle, 0); + assert_eq!(got.len(), 4, "the whole set, from anywhere in it"); + + let main = &got[0]; + let (mflags, mpath) = (main.flags, &main.path); + assert_ne!(mflags & GIT_WORKTREE_MAIN, 0); + assert_eq!(mpath, dir.to_str().unwrap(), "the main worktree's path"); + assert_eq!(mflags & GIT_WORKTREE_CURRENT, 0, "we are not in main"); + + let current: Vec<_> = got + .iter() + .filter(|t| t.flags & GIT_WORKTREE_CURRENT != 0) + .collect(); + assert_eq!(current.len(), 1, "exactly one CURRENT"); + assert_eq!( + current[0].path, + feature.to_str().unwrap(), + "the one we opened" + ); +} + +/// A worktree whose checkout was deleted behind git's back is still an +/// administrative entry. It is reported as `PRUNABLE` rather than dropped: +/// a row a client cannot navigate to is the thing it most needs told. +#[test] +fn worktrees_reports_a_deleted_checkout_as_prunable() { + let (dir, trees) = worktree_fixture(); + std::fs::remove_dir_all(trees.join("feature")).unwrap(); + let (handle, _info) = open(dir.to_str().unwrap()).unwrap(); + let (_flags, got) = worktrees_of(&handle, 0); + assert_eq!(got.len(), 4, "still listed, not silently dropped"); + let gone = got + .iter() + .find(|t| t.path == trees.join("feature").to_str().unwrap()) + .expect("the removed worktree is still an entry"); + assert_ne!(gone.flags & GIT_WORKTREE_PRUNABLE, 0, "PRUNABLE"); + // git agrees it is prunable. + assert!( + git_out(&dir, &["worktree", "list", "--porcelain"]).contains("prunable"), + "git also calls it prunable" + ); +} + +/// `after_pos` skips what a previous page delivered, in the same order. +/// Does not exercise the budget itself: `worktrees_max` comes from +/// process-global env (256 by default) and no fixture here approaches it, +/// so `TRUNCATED` and its trailing `CURSOR` are covered by inspection only. +#[test] +fn worktrees_resumes_from_a_cursor() { + let (dir, trees) = worktree_fixture(); + let (handle, _info) = open(dir.to_str().unwrap()).unwrap(); + let (_flags, all) = worktrees_of(&handle, 0); + assert_eq!(all.len(), 4); + // Resume past the main record: the rest of the set, in the same order. + let (_flags, rest) = worktrees_of(&handle, 1); + assert_eq!(rest.len(), 3, "the main record was skipped"); + assert_eq!(rest, all[1..].to_vec(), "same records, same order"); + assert!( + rest.iter().all(|t| t.flags & GIT_WORKTREE_MAIN == 0), + "and none of them is main" + ); + let _ = trees; +} + +/// Unknown request flags are refused rather than ignored, so a future bit +/// cannot be silently dropped by an old server. +#[test] +fn worktrees_refuses_unknown_flags() { + let dir = fixture(); + let (handle, _info) = open(dir.to_str().unwrap()).unwrap(); + let resp = handle.worktrees( + &GitWorktreesRequest { + nonce: 3, + repo_id: 0, + flags: 0x80, + after_pos: 0, + }, + &Cancel::default(), + ); + let (nonce, status, _flags, _records) = parse_git_worktrees_resp(&resp).unwrap(); + assert_eq!((nonce, status), (3, GIT_STATUS_INVALID)); +} + +/// The `WORKTREE_GEN` state record is what makes the list live: adding, +/// removing or locking a worktree leaves every ref and status record +/// identical, so without a generation in the snapshot the engine's +/// identical-snapshot suppression would swallow the push. +#[test] +fn worktree_gen_moves_on_add_remove_and_lock() { + let gen_of = |dir: &Path| { + let (handle, _info) = open(dir.to_str().unwrap()).unwrap(); + let msg = wait_first_state(&handle, StateOptions::default()); + let (_id, _sid, _flags, records) = parse_git_state(&msg).unwrap(); + let found: Vec<_> = git_state_records(&records) + .filter_map(|r| match r { + GitStateRecord::WorktreeGen { count, digest } => Some((count, digest)), + _ => None, + }) + .collect(); + assert_eq!(found.len(), 1, "exactly one WORKTREE_GEN per snapshot"); + found[0] + }; + + let dir = temp_dir(); + git(&dir, &["init", "-b", "main"]); + std::fs::write(dir.join("a.txt"), "one\n").unwrap(); + git(&dir, &["add", "."]); + git(&dir, &["commit", "-m", "first"]); + // A repository with no linked worktrees still has the main one. + let bare_of_worktrees = gen_of(&dir); + assert_eq!(bare_of_worktrees.0, 1, "the main worktree counts"); + + let trees = temp_dir(); + let one = trees.join("one"); + git( + &dir, + &["worktree", "add", "-b", "one", one.to_str().unwrap()], + ); + let added = gen_of(&dir); + assert_eq!(added.0, 2, "count follows the set"); + assert_ne!(added.1, bare_of_worktrees.1, "digest moves on add"); + + // Locking changes no ref and no file git tracks — the case a + // refetch-on-ref-move client would miss entirely. + git(&dir, &["worktree", "lock", one.to_str().unwrap()]); + let locked = gen_of(&dir); + assert_eq!(locked.0, added.0, "still two worktrees"); + assert_ne!(locked.1, added.1, "digest moves on lock"); + git(&dir, &["worktree", "unlock", one.to_str().unwrap()]); + assert_eq!(gen_of(&dir).1, added.1, "and back on unlock"); + + // Removing it takes no ref with it: the branch survives. + git(&dir, &["worktree", "remove", one.to_str().unwrap()]); + let removed = gen_of(&dir); + assert_eq!(removed, bare_of_worktrees, "back to the original set"); + assert_eq!( + git_out(&dir, &["rev-parse", "--verify", "refs/heads/one"]).len(), + 40, + "while the branch it created is untouched — which is why a \ + ref-move refetch cannot see the removal" + ); +} diff --git a/crates/guest/README.md b/crates/guest/README.md index b6f66520..d9b40c73 100644 --- a/crates/guest/README.md +++ b/crates/guest/README.md @@ -52,6 +52,16 @@ and updates for unknown PTYs are safely discarded, unrelated packets stay in the client's bounded pending queue, and dropping an update token sends no ACK. See `examples/terminal_subscription.rs`. +A guest reaches the host only through Blit packets, so an extension that must +observe the machine spawns a native child with the process family +(`blit_guest::remote::process`) and reads its stdout. The `systemd` extension +(`extensions/systemd`) does this end to end: it keeps the unit tables live from +`systemctl`, poked by D-Bus unit signals, and publishes snapshots and deltas as +JSON on the `blit.systemd.v1` channel while serving +`@systemd list|get|watch|status`. It also shows a single-threaded loop that +multiplexes process output, channel subscribers, and command invocations over +one endpoint, which the blocking typed helpers cannot do on their own. + `EventLoop` combines packet dispatch with one-shot monotonic timers. It keeps callbacks in a guest-side min-heap, passes only the nearest deadline to the host wait call, gives a ready packet priority over simultaneous timers, and runs all diff --git a/crates/guest/src/channel.rs b/crates/guest/src/channel.rs index a923397a..ec369e85 100644 --- a/crates/guest/src/channel.rs +++ b/crates/guest/src/channel.rs @@ -178,7 +178,30 @@ impl Listener { let packet = client .recv_matching(|packet| listener_packet(packet, self.id))? .ok_or(ClientError::EndpointClosed)?; - match wire::parse_channel_message(&packet)? { + self.interpret(&packet) + } + + /// Interpret a packet the caller already read, without blocking. + /// + /// `Ok(None)` means the packet is not this listener's, so the caller should + /// route it elsewhere. An extension that also has to watch timers or other + /// families cannot use [`accept`](Self::accept): it blocks until *its* + /// packet arrives while everything else queues behind it, so a backoff timer + /// never fires and a process exit is never seen. Such a caller owns the + /// receive loop (`Client::wait_until` then `Client::recv`) and routes each + /// packet here. + pub fn offer(&mut self, packet: &[u8]) -> Result, Error> { + if self.closed { + return Err(Error::Closed); + } + if !listener_packet(packet, self.id) { + return Ok(None); + } + self.interpret(packet).map(Some) + } + + fn interpret(&mut self, packet: &[u8]) -> Result { + match wire::parse_channel_message(packet)? { Some(wire::ChannelMessage::Accepted { channel_id, listener_id, @@ -330,7 +353,29 @@ impl Channel { let packet = client .recv_matching(|packet| connected_packet(packet, self.id))? .ok_or(ClientError::EndpointClosed)?; - match wire::parse_channel_message(&packet)? { + self.interpret(&packet) + } + + /// Interpret a packet the caller already read, without blocking. + /// + /// `Ok(None)` means the packet is not this channel's. See + /// [`Listener::offer`] for why a supervising extension needs this rather + /// than [`receive`](Self::receive). + pub fn offer(&mut self, packet: &[u8]) -> Result, Error> { + if self.closed { + return Err(Error::Closed); + } + if self.pending_delivery.is_some() { + return Err(Error::DeliveryPending); + } + if !connected_packet(packet, self.id) { + return Ok(None); + } + self.interpret(packet).map(Some) + } + + fn interpret(&mut self, packet: &[u8]) -> Result { + match wire::parse_channel_message(packet)? { Some(wire::ChannelMessage::Data { channel_id, payload, diff --git a/crates/guest/src/command.rs b/crates/guest/src/command.rs index cad62f4e..bc91d59a 100644 --- a/crates/guest/src/command.rs +++ b/crates/guest/src/command.rs @@ -192,6 +192,34 @@ impl CommandProvider { } } + /// Interpret a packet the caller already read, without waiting for one. + /// + /// `Ok(None)` means the packet was not for this provider's listener, so the + /// caller should route it elsewhere. This is what an extension needs when it + /// must also service timers or other families: [`accept`](Self::accept) + /// blocks until an invocation arrives, so a backoff deadline never comes due + /// and a `PROCESS_EXIT` sits unread in the client's pending queue. + /// + /// One caveat, deliberate: once the listener yields a channel this still + /// blocks for that channel's `INVOKE`, because an `Invocation` is not a + /// meaningful value without one. That wait is bounded by a single round trip + /// from a peer which has already opened the channel and, per `blit.cli.v1`, + /// sends `INVOKE` as its first message — unlike `accept`, which waits on a + /// peer that may never appear at all. + pub fn offer( + &mut self, + client: &mut Client, + packet: &[u8], + ) -> Result, Error> { + match self.listener.offer(packet)? { + Some(ListenerEvent::Accepted(channel)) => Invocation::begin(client, channel) + .map(ProviderEvent::Invocation) + .map(Some), + Some(ListenerEvent::Closed(closed)) => Ok(Some(ProviderEvent::Closed(closed))), + None => Ok(None), + } + } + /// Unregister and close the listener. The close still runs if unregister /// fails, so a stale advertisement cannot keep accepting invocations. pub fn close(&mut self, client: &mut Client) -> Result<(), Error> { diff --git a/crates/remote/src/channel.rs b/crates/remote/src/channel.rs index dbbd67b5..855f174e 100644 --- a/crates/remote/src/channel.rs +++ b/crates/remote/src/channel.rs @@ -10,16 +10,26 @@ use std::fmt; pub const CHANNEL: u8 = 0x95; /// `S2C_HELLO` feature bit for native channels. pub const FEATURE_CHANNEL: u32 = 1 << 12; +/// `S2C_HELLO` feature bit for `CHANNEL_WATCH`. +/// +/// Its own bit rather than `FEATURE_CHANNEL`, because a `WATCH` an older +/// server does not know is an unknown sub-operation: the family's skip rule +/// drops it without a reply, which is indistinguishable from a name nobody +/// serves. A client that cannot see this bit has to keep probing by connect. +pub const FEATURE_CHANNEL_WATCH: u32 = 1 << 26; pub const CHANNEL_LISTEN: u8 = 1; pub const CHANNEL_CONNECT: u8 = 2; pub const CHANNEL_DATA: u8 = 3; pub const CHANNEL_ACK: u8 = 4; pub const CHANNEL_CLOSE: u8 = 5; +pub const CHANNEL_WATCH: u8 = 6; +pub const CHANNEL_UNWATCH: u8 = 7; pub const CHANNEL_OPENED: u8 = 1; pub const CHANNEL_ACCEPTED: u8 = 2; pub const CHANNEL_CLOSED: u8 = 5; +pub const CHANNEL_NAMES: u8 = 6; /// `CONNECT.flags`: require the named listener to have this exact generation. pub const CHANNEL_EXPECT_LISTENER_TOKEN: u8 = 1 << 0; @@ -37,6 +47,13 @@ pub const CHANNEL_MAX_PAYLOAD: usize = 1024 * 1024; pub const CHANNEL_MAX_DETAIL: usize = 4 * 1024; pub const CHANNEL_WINDOW_BYTES: u64 = 1024 * 1024; pub const CHANNEL_MAX_UNCONSUMED_MESSAGES: usize = 1024; +/// Names one `CHANNEL_WATCH` may declare. +/// +/// A watch names what it cares about instead of asking for the whole registry: +/// the reply is then bounded by the request, and the transient +/// `blit.cli..` listeners every extension mints cannot make a +/// watcher's traffic scale with churn it has no interest in. +pub const CHANNEL_MAX_WATCH_NAMES: usize = 32; /// One decoded client-to-server channel operation. #[derive(Clone, Debug, PartialEq, Eq)] @@ -64,6 +81,15 @@ pub enum ChannelRequest<'a> { channel_id: u32, reason: u8, }, + /// Follow which of `names` currently have a listener, on a client-created + /// ID that carries no stream and never accepts. + Watch { + channel_id: u32, + names: Vec<&'a str>, + }, + Unwatch { + channel_id: u32, + }, } impl ChannelRequest<'_> { @@ -73,7 +99,9 @@ impl ChannelRequest<'_> { | Self::Connect { channel_id, .. } | Self::Data { channel_id, .. } | Self::Ack { channel_id, .. } - | Self::Close { channel_id, .. } => *channel_id, + | Self::Close { channel_id, .. } + | Self::Watch { channel_id, .. } + | Self::Unwatch { channel_id } => *channel_id, } } } @@ -109,6 +137,14 @@ pub enum ChannelMessage<'a> { reason: u8, detail: &'a str, }, + /// Which of a watch's declared names have a listener right now, in the + /// order they were declared. A name the client asked about and does not + /// find here has no listener; absence is the whole answer, so an empty + /// list is meaningful rather than a no-op. + Names { + channel_id: u32, + names: Vec<&'a str>, + }, } /// Structural or version-1 validation failure in a known channel operation. @@ -124,6 +160,8 @@ pub enum ChannelDecodeError { EmptyPayload, TooLarge, InvalidCloseReason, + EmptyWatch, + DuplicateWatchName, } impl fmt::Display for ChannelDecodeError { @@ -139,6 +177,8 @@ impl fmt::Display for ChannelDecodeError { Self::EmptyPayload => "channel data payload is empty", Self::TooLarge => "channel field exceeds its size limit", Self::InvalidCloseReason => "client channel close reason is invalid", + Self::EmptyWatch => "channel watch declares no names", + Self::DuplicateWatchName => "channel watch names must be distinct", }) } } @@ -285,6 +325,36 @@ pub fn parse_channel_request( reason: body[0], })) } + CHANNEL_WATCH => { + if channel_id & 1 != 0 { + return Err(ChannelDecodeError::InvalidClientId); + } + let names = decode_name_list(body, CHANNEL_MAX_WATCH_NAMES)?; + if names.is_empty() { + return Err(ChannelDecodeError::EmptyWatch); + } + // A repeated name would appear twice in a reply whose whole + // meaning is which names are claimed, so the ambiguity is refused + // rather than carried. The list is short enough that the obvious + // scan is cheaper than a set. + if names + .iter() + .enumerate() + .any(|(index, name)| names[..index].contains(name)) + { + return Err(ChannelDecodeError::DuplicateWatchName); + } + Ok(Some(ChannelRequest::Watch { channel_id, names })) + } + CHANNEL_UNWATCH => { + if channel_id & 1 != 0 { + return Err(ChannelDecodeError::InvalidClientId); + } + if !body.is_empty() { + return Err(ChannelDecodeError::TrailingBytes); + } + Ok(Some(ChannelRequest::Unwatch { channel_id })) + } _ => Ok(None), } } @@ -372,6 +442,10 @@ pub fn parse_channel_message( detail, })) } + CHANNEL_NAMES => Ok(Some(ChannelMessage::Names { + channel_id, + names: decode_name_list(body, CHANNEL_MAX_WATCH_NAMES)?, + })), _ => Ok(None), } } @@ -495,6 +569,32 @@ pub fn msg_channel_accepted( Some(msg) } +pub fn msg_channel_watch(channel_id: u32, names: &[&str]) -> Option> { + if channel_id & 1 != 0 || names.is_empty() { + return None; + } + if names + .iter() + .enumerate() + .any(|(index, name)| names[..index].contains(name)) + { + return None; + } + encode_name_list(CHANNEL_WATCH, channel_id, names) +} + +pub fn msg_channel_unwatch(channel_id: u32) -> Option> { + if channel_id & 1 != 0 { + return None; + } + Some(envelope(CHANNEL_UNWATCH, channel_id, 0)) +} + +/// The names of a watch that currently have a listener, in declared order. +pub fn msg_channel_names(channel_id: u32, names: &[&str]) -> Option> { + encode_name_list(CHANNEL_NAMES, channel_id, names) +} + pub fn msg_channel_closed(channel_id: u32, reason: u8, detail: &str) -> Option> { if detail.len() > CHANNEL_MAX_DETAIL { return None; @@ -513,6 +613,65 @@ fn envelope(kind: u8, channel_id: u32, body_capacity: usize) -> Vec { msg } +/// `[flags:1][count:2]` then `count` × `[len:2][name]`, exactly. +/// +/// Both directions of a watch carry a name list, and both need it to be +/// self-describing: the reply repeats the names rather than answering with a +/// bitmap over the request, so a client reading it needs no memory of what it +/// asked and a packet on the wire can be read on its own. +fn encode_name_list(kind: u8, channel_id: u32, names: &[&str]) -> Option> { + if names.len() > CHANNEL_MAX_WATCH_NAMES { + return None; + } + let count = u16::try_from(names.len()).ok()?; + let bytes: usize = names.iter().map(|name| 2 + name.len()).sum(); + let mut msg = envelope(kind, channel_id, 3 + bytes); + msg.push(0); + msg.extend_from_slice(&count.to_le_bytes()); + for name in names { + if decode_name(name.as_bytes()).is_err() { + return None; + } + msg.extend_from_slice(&(name.len() as u16).to_le_bytes()); + msg.extend_from_slice(name.as_bytes()); + } + Some(msg) +} + +fn decode_name_list(body: &[u8], limit: usize) -> Result, ChannelDecodeError> { + if body.len() < 3 { + return Err(ChannelDecodeError::Truncated); + } + if body[0] != 0 { + return Err(ChannelDecodeError::InvalidFlags); + } + let count = u16::from_le_bytes([body[1], body[2]]) as usize; + if count > limit { + return Err(ChannelDecodeError::TooLarge); + } + let mut names = Vec::with_capacity(count); + let mut offset: usize = 3; + for _ in 0..count { + let length_end = offset.checked_add(2).ok_or(ChannelDecodeError::TooLarge)?; + if body.len() < length_end { + return Err(ChannelDecodeError::Truncated); + } + let length = u16::from_le_bytes([body[offset], body[offset + 1]]) as usize; + let name_end = length_end + .checked_add(length) + .ok_or(ChannelDecodeError::TooLarge)?; + if body.len() < name_end { + return Err(ChannelDecodeError::Truncated); + } + names.push(decode_name(&body[length_end..name_end])?); + offset = name_end; + } + if offset != body.len() { + return Err(ChannelDecodeError::TrailingBytes); + } + Ok(names) +} + fn decode_name(bytes: &[u8]) -> Result<&str, ChannelDecodeError> { if bytes.is_empty() || bytes.len() > CHANNEL_MAX_NAME { return Err(ChannelDecodeError::InvalidName); @@ -681,6 +840,95 @@ mod tests { ); } + #[test] + fn watch_round_trip() { + let wire = msg_channel_watch(6, &["blit.session.v1", "blit.systemd.v1"]).unwrap(); + assert_eq!( + parse_channel_request(&wire).unwrap(), + Some(ChannelRequest::Watch { + channel_id: 6, + names: vec!["blit.session.v1", "blit.systemd.v1"], + }) + ); + let unwatch = msg_channel_unwatch(6).unwrap(); + assert_eq!( + parse_channel_request(&unwatch).unwrap(), + Some(ChannelRequest::Unwatch { channel_id: 6 }) + ); + } + + #[test] + fn names_round_trip_including_the_empty_answer() { + let wire = msg_channel_names(6, &["blit.systemd.v1"]).unwrap(); + assert_eq!( + parse_channel_message(&wire).unwrap(), + Some(ChannelMessage::Names { + channel_id: 6, + names: vec!["blit.systemd.v1"], + }) + ); + // Nothing claimed is an answer, not a packet to withhold. + let empty = msg_channel_names(6, &[]).unwrap(); + assert_eq!( + parse_channel_message(&empty).unwrap(), + Some(ChannelMessage::Names { + channel_id: 6, + names: vec![], + }) + ); + } + + #[test] + fn watch_rejects_what_it_cannot_answer() { + assert_eq!(msg_channel_watch(6, &[]), None); + assert_eq!(msg_channel_watch(7, &["a"]), None); + assert_eq!(msg_channel_watch(6, &["a", "a"]), None); + assert_eq!( + parse_channel_request(&[CHANNEL, CHANNEL_WATCH, 6, 0, 0, 0, 0, 0, 0]), + Err(ChannelDecodeError::EmptyWatch) + ); + let duplicated = [ + CHANNEL, + CHANNEL_WATCH, + 6, + 0, + 0, + 0, + 0, + 2, + 0, + 1, + 0, + b'a', + 1, + 0, + b'a', + ]; + assert_eq!( + parse_channel_request(&duplicated), + Err(ChannelDecodeError::DuplicateWatchName) + ); + // An odd ID is server-created, and the server never watches. + assert_eq!( + parse_channel_request(&[CHANNEL, CHANNEL_UNWATCH, 7, 0, 0, 0]), + Err(ChannelDecodeError::InvalidClientId) + ); + assert_eq!( + parse_channel_request(&[CHANNEL, CHANNEL_UNWATCH, 6, 0, 0, 0, 1]), + Err(ChannelDecodeError::TrailingBytes) + ); + // A count that outruns the body is truncation, not an empty list. + assert_eq!( + parse_channel_request(&[CHANNEL, CHANNEL_WATCH, 6, 0, 0, 0, 0, 1, 0]), + Err(ChannelDecodeError::Truncated) + ); + // Reserved flags stay reserved, so a future bit cannot be eaten. + assert_eq!( + parse_channel_message(&[CHANNEL, CHANNEL_NAMES, 6, 0, 0, 0, 1, 0, 0]), + Err(ChannelDecodeError::InvalidFlags) + ); + } + #[test] fn unknown_kinds_are_skipped() { let wire = [CHANNEL, 99, 2, 0, 0, 0, 1, 2, 3]; @@ -723,7 +971,14 @@ mod tests { | crate::FEATURE_SURFACE_TOUCH | crate::FEATURE_SURFACE_TEXT_INPUT | crate::FEATURE_CLIENT_CONTROL - | crate::desktop::FEATURE_DESKTOP; + | crate::desktop::FEATURE_DESKTOP + | crate::media::FEATURE_DESKTOP_MEDIA + | crate::process::FEATURE_PROCESS + | crate::process::FEATURE_PROCESS_SESSION_ENV + | crate::process::FEATURE_APP_SOCKET + | crate::env::FEATURE_ENV + | crate::extension::FEATURE_EXTENSION; assert_eq!(FEATURE_CHANNEL & taken, 0); + assert_eq!(FEATURE_CHANNEL_WATCH & (taken | FEATURE_CHANNEL), 0); } } diff --git a/crates/remote/src/env.rs b/crates/remote/src/env.rs new file mode 100644 index 00000000..376db39c --- /dev/null +++ b/crates/remote/src/env.rs @@ -0,0 +1,383 @@ +//! Server environment wire protocol (docs/design/env.md). +//! +//! One request, one reply: a client asks for the server's environment and gets +//! every variable back, sorted by key. It exists because a client cannot +//! otherwise learn anything about the session it is attached to — not the +//! compositor socket, not `XDG_DATA_DIRS`, nothing — and an extension has no +//! host access at all beyond this protocol. +//! +//! This exposes whatever the server was started with, credentials included. +//! `BLIT_ENV=0` refuses the whole family at dispatch with `PERMISSION`; see the +//! security section of the design doc before widening anything here. +//! +//! All integers little-endian, tightly packed, as everywhere in the protocol. + +use std::collections::BTreeMap; + +/// Request the server environment: [0x75][nonce:2] +pub const C2S_ENV_GET: u8 = 0x75; + +/// The environment: [0x75][nonce:2][status:1][count:2] then `count` records of +/// [key_len:2][key:N][value_len:4][value:N], ascending by key. +pub const S2C_ENV: u8 = 0x75; + +/// `S2C_HELLO` feature bit: server answers `ENV_*`. `BLIT_ENV=0` refuses at +/// dispatch with `PERMISSION` rather than un-advertising, matching `FEATURE_KV`, +/// so a client can tell "this server has no such family" from "the operator +/// turned it off". +pub const FEATURE_ENV: u32 = 1 << 24; + +/// Longest key accepted, in bytes. +pub const ENV_MAX_KEY: usize = 4 * 1024; +/// Longest value accepted, in bytes. +pub const ENV_MAX_VALUE: usize = 1024 * 1024; +/// Most variables one reply may carry. +pub const ENV_MAX_COUNT: usize = 8192; +/// Cap on the sum of all key and value bytes in one reply. +pub const ENV_MAX_TOTAL: usize = 4 * 1024 * 1024; + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum EnvCodecError { + /// Truncated, trailing, or otherwise malformed. + Invalid, + /// A documented limit was exceeded. + TooLarge, +} + +impl EnvCodecError { + pub fn status(self) -> u8 { + match self { + Self::Invalid => crate::STATUS_INVALID, + Self::TooLarge => crate::STATUS_TOO_LARGE, + } + } +} + +pub fn is_c2s_env(opcode: u8) -> bool { + opcode == C2S_ENV_GET +} + +/// Encode `C2S_ENV_GET`. +pub fn msg_env_get(nonce: u16) -> Vec { + let mut msg = Vec::with_capacity(3); + msg.push(C2S_ENV_GET); + msg.extend_from_slice(&nonce.to_le_bytes()); + msg +} + +/// Decode `C2S_ENV_GET`, returning the nonce. +pub fn parse_env_get(msg: &[u8]) -> Result { + let body = body_of(msg, C2S_ENV_GET)?; + if body.len() != 2 { + return Err(EnvCodecError::Invalid); + } + Ok(u16::from_le_bytes([body[0], body[1]])) +} + +/// Encode `S2C_ENV`. Entries are emitted in `BTreeMap` order, so the reply is +/// byte-identical for an unchanged environment. +/// +/// Keys and values are raw bytes: a Unix environment is not required to be +/// UTF-8, and silently dropping a non-UTF-8 entry would be a worse answer than +/// handing it over as it is. +pub fn msg_env( + nonce: u16, + status: u8, + entries: &BTreeMap, Vec>, +) -> Result, EnvCodecError> { + if entries.len() > ENV_MAX_COUNT { + return Err(EnvCodecError::TooLarge); + } + let mut total = 0usize; + for (key, value) in entries { + if key.is_empty() || key.len() > ENV_MAX_KEY || value.len() > ENV_MAX_VALUE { + return Err(EnvCodecError::TooLarge); + } + // A NUL cannot survive a round trip through execve, so refuse to claim + // it did. + if key.contains(&0) || value.contains(&0) { + return Err(EnvCodecError::Invalid); + } + total = total + .checked_add(key.len()) + .and_then(|sum| sum.checked_add(value.len())) + .ok_or(EnvCodecError::TooLarge)?; + if total > ENV_MAX_TOTAL { + return Err(EnvCodecError::TooLarge); + } + } + let mut msg = Vec::with_capacity(6 + total + entries.len() * 6); + msg.push(S2C_ENV); + msg.extend_from_slice(&nonce.to_le_bytes()); + msg.push(status); + msg.extend_from_slice(&(entries.len() as u16).to_le_bytes()); + for (key, value) in entries { + msg.extend_from_slice(&(key.len() as u16).to_le_bytes()); + msg.extend_from_slice(key); + msg.extend_from_slice(&(value.len() as u32).to_le_bytes()); + msg.extend_from_slice(value); + } + Ok(msg) +} + +/// A decoded `S2C_ENV`. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct EnvReply { + pub nonce: u16, + pub status: u8, + pub entries: BTreeMap, Vec>, +} + +/// Decode `S2C_ENV`. +pub fn parse_env(msg: &[u8]) -> Result { + let mut body = body_of(msg, S2C_ENV)?; + let nonce = take_u16(&mut body)?; + let status = take_u8(&mut body)?; + let count = take_u16(&mut body)? as usize; + if count > ENV_MAX_COUNT { + return Err(EnvCodecError::TooLarge); + } + let mut entries = BTreeMap::new(); + let mut total = 0usize; + for _ in 0..count { + let key_len = take_u16(&mut body)? as usize; + if key_len == 0 || key_len > ENV_MAX_KEY { + return Err(EnvCodecError::TooLarge); + } + let key = take_bytes(&mut body, key_len)?.to_vec(); + let value_len = take_u32(&mut body)? as usize; + if value_len > ENV_MAX_VALUE { + return Err(EnvCodecError::TooLarge); + } + let value = take_bytes(&mut body, value_len)?.to_vec(); + total = total + .checked_add(key_len) + .and_then(|sum| sum.checked_add(value_len)) + .ok_or(EnvCodecError::TooLarge)?; + if total > ENV_MAX_TOTAL { + return Err(EnvCodecError::TooLarge); + } + // A duplicate key would silently lose one of the two values, so it is + // malformed rather than merged. + if entries.insert(key, value).is_some() { + return Err(EnvCodecError::Invalid); + } + } + if !body.is_empty() { + return Err(EnvCodecError::Invalid); + } + Ok(EnvReply { + nonce, + status, + entries, + }) +} + +fn body_of(msg: &[u8], opcode: u8) -> Result<&[u8], EnvCodecError> { + match msg.split_first() { + Some((&first, rest)) if first == opcode => Ok(rest), + _ => Err(EnvCodecError::Invalid), + } +} + +fn take_bytes<'a>(body: &mut &'a [u8], len: usize) -> Result<&'a [u8], EnvCodecError> { + if body.len() < len { + return Err(EnvCodecError::Invalid); + } + let (head, rest) = body.split_at(len); + *body = rest; + Ok(head) +} + +fn take_u8(body: &mut &[u8]) -> Result { + Ok(take_bytes(body, 1)?[0]) +} + +fn take_u16(body: &mut &[u8]) -> Result { + let bytes = take_bytes(body, 2)?; + Ok(u16::from_le_bytes([bytes[0], bytes[1]])) +} + +fn take_u32(body: &mut &[u8]) -> Result { + let bytes = take_bytes(body, 4)?; + Ok(u32::from_le_bytes([bytes[0], bytes[1], bytes[2], bytes[3]])) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn sample() -> BTreeMap, Vec> { + BTreeMap::from([ + (b"HOME".to_vec(), b"/home/pcarrier".to_vec()), + (b"XDG_DATA_DIRS".to_vec(), b"/usr/share:/usr/local".to_vec()), + // Empty values are legal and must survive. + (b"EMPTY".to_vec(), Vec::new()), + ]) + } + + #[test] + fn allocation_is_locked() { + assert_eq!(C2S_ENV_GET, 0x75); + assert_eq!(S2C_ENV, 0x75); + assert_eq!(FEATURE_ENV, 1 << 24); + } + + #[test] + fn a_request_round_trips() { + assert_eq!(parse_env_get(&msg_env_get(0x1234)), Ok(0x1234)); + // Trailing bytes are refused, not ignored. + let mut trailing = msg_env_get(1); + trailing.push(0); + assert_eq!(parse_env_get(&trailing), Err(EnvCodecError::Invalid)); + } + + #[test] + fn a_reply_round_trips_and_is_deterministic() { + let entries = sample(); + let encoded = msg_env(7, crate::STATUS_OK, &entries).expect("encodes"); + let decoded = parse_env(&encoded).expect("decodes"); + assert_eq!(decoded.nonce, 7); + assert_eq!(decoded.status, crate::STATUS_OK); + assert_eq!(decoded.entries, entries); + // Same input, same bytes — the ordering is part of the contract. + assert_eq!( + encoded, + msg_env(7, crate::STATUS_OK, &sample()).expect("encodes") + ); + } + + /// A refusal carries no entries but still answers under the nonce, so a + /// client waiting on one is never left hanging. + #[test] + fn a_refusal_is_a_well_formed_empty_reply() { + let encoded = msg_env(9, crate::STATUS_PERMISSION, &BTreeMap::new()).expect("encodes"); + let decoded = parse_env(&encoded).expect("decodes"); + assert_eq!(decoded.status, crate::STATUS_PERMISSION); + assert!(decoded.entries.is_empty()); + assert_eq!(decoded.nonce, 9); + } + + #[test] + fn malformed_replies_are_refused() { + // Truncated mid-record. + let encoded = msg_env(1, crate::STATUS_OK, &sample()).expect("encodes"); + for cut in 1..encoded.len() { + assert!( + parse_env(&encoded[..cut]).is_err(), + "prefix of length {cut} must not decode" + ); + } + // Wrong opcode. + let mut wrong = encoded.clone(); + wrong[0] = 0x76; + assert_eq!(parse_env(&wrong), Err(EnvCodecError::Invalid)); + // Trailing bytes. + let mut trailing = encoded; + trailing.push(0xFF); + assert_eq!(parse_env(&trailing), Err(EnvCodecError::Invalid)); + } + + #[test] + fn a_nul_is_refused_rather_than_claimed_to_round_trip() { + let with_nul = BTreeMap::from([(b"K".to_vec(), b"a\0b".to_vec())]); + assert_eq!( + msg_env(1, crate::STATUS_OK, &with_nul), + Err(EnvCodecError::Invalid) + ); + let empty_key = BTreeMap::from([(Vec::new(), b"v".to_vec())]); + assert_eq!( + msg_env(1, crate::STATUS_OK, &empty_key), + Err(EnvCodecError::TooLarge) + ); + } + + #[test] + fn an_oversized_value_is_refused() { + let big = BTreeMap::from([(b"K".to_vec(), vec![b'x'; ENV_MAX_VALUE + 1])]); + assert_eq!( + msg_env(1, crate::STATUS_OK, &big), + Err(EnvCodecError::TooLarge) + ); + } + + /// Stamped identity and the socket request that mints it. They live here + /// rather than in `lib.rs`'s test module only because that module is 3000 + /// lines away from anything related. + #[test] + fn app_socket_and_surface_origin_round_trip() { + use crate::{ + msg_app_socket_reply, msg_app_socket_request, msg_surface_origin, + parse_app_socket_reply, parse_app_socket_request, parse_server_msg, + }; + + let request = msg_app_socket_request(11, "legcord", "a1b2"); + assert_eq!( + parse_app_socket_request(&request), + Some((11, "legcord", "a1b2")) + ); + + let reply = msg_app_socket_reply(11, crate::STATUS_OK, "blit-app-legcord-a1b2"); + assert_eq!( + parse_app_socket_reply(&reply), + Some((11, crate::STATUS_OK, "blit-app-legcord-a1b2")) + ); + // A refusal carries an empty name, and must still parse. + let refused = msg_app_socket_reply(11, crate::STATUS_INVALID, ""); + assert_eq!( + parse_app_socket_reply(&refused), + Some((11, crate::STATUS_INVALID, "")) + ); + + let origin = msg_surface_origin(7, "blit", "legcord", "a1b2"); + match parse_server_msg(&origin) { + Some(crate::ServerMsg::SurfaceOrigin { + surface_id, + sandbox_engine, + app_id, + instance_id, + }) => { + assert_eq!( + (surface_id, sandbox_engine, app_id, instance_id), + (7, "blit", "legcord", "a1b2") + ); + } + _ => panic!("SURFACE_ORIGIN did not decode as SurfaceOrigin"), + } + + // Truncation must fail the parse rather than read a field as empty — + // an empty app_id would silently attribute a window to nobody. + for cut in 1..origin.len() { + let partial = &origin[..cut]; + assert!( + !matches!( + parse_server_msg(partial), + Some(crate::ServerMsg::SurfaceOrigin { .. }) + ), + "prefix of length {cut} must not decode as SurfaceOrigin" + ); + } + let mut trailing = origin; + trailing.push(0); + assert!(!matches!( + parse_server_msg(&trailing), + Some(crate::ServerMsg::SurfaceOrigin { .. }) + )); + } + + /// A duplicate key on the wire would silently drop one value. + #[test] + fn a_duplicate_key_is_malformed() { + let mut msg = vec![S2C_ENV]; + msg.extend_from_slice(&1u16.to_le_bytes()); + msg.push(crate::STATUS_OK); + msg.extend_from_slice(&2u16.to_le_bytes()); + for value in [b"one", b"two"] { + msg.extend_from_slice(&1u16.to_le_bytes()); + msg.push(b'K'); + msg.extend_from_slice(&3u32.to_le_bytes()); + msg.extend_from_slice(value); + } + assert_eq!(parse_env(&msg), Err(EnvCodecError::Invalid)); + } +} diff --git a/crates/remote/src/fs.rs b/crates/remote/src/fs.rs index 8f211745..890464e8 100644 --- a/crates/remote/src/fs.rs +++ b/crates/remote/src/fs.rs @@ -32,7 +32,7 @@ pub const C2S_FS_OP: u8 = 0x45; pub const C2S_FS_SEARCH: u8 = 0x46; /// Fetch the candidate file list under a root (no sync), for client-side /// fuzzy search (docs/design/fs-search.md): [0x47][nonce:2][flags:1][root_len:2][root:N]. -/// `flags` is reserved; nonzero answers `INVALID`. +/// `flags` is `FS_INDEX_DIRS_ONLY`; unknown bits answer `INVALID`. pub const C2S_FS_INDEX: u8 = 0x47; /// Content search under a root (no sync), docs/design/fs-grep.md: @@ -58,6 +58,25 @@ pub const C2S_FS_UPLOAD_FINISH: u8 = 0x4B; /// Abort the upload: [0x4C][upload_id:2]. No reply. pub const C2S_FS_UPLOAD_CANCEL: u8 = 0x4C; +/// Read whole files without a sync (docs/design/fs-read.md): +/// [0x4D][nonce:2][flags:1][max_bytes:4][group_count:2] then group_count × +/// ( [path_count:2] then path_count × [path_len:2][path:N] ). +/// +/// The family's one-shot read. `FS_FETCH` needs an established sync — a watched +/// tree, which is the wrong shape for reading a fixed set of files once — so +/// everything that wanted a handful of files had to sync a directory it did not +/// want watched, or shell out to `cat`. Paths are absolute and independent; +/// nothing is watched and no state is kept. +/// +/// `max_bytes` is the per-file ceiling, zero meaning [`FS_READ_DEFAULT_BYTES`]; +/// a larger file is reported rather than read, so a caller drawing 2em tiles +/// does not have a theme's megabyte SVG pushed at it. `flags` is `FS_READ_FIRST`. +/// +/// Paths are grouped, and a group is one question: with `FS_READ_FIRST` each +/// group is answered by its own first readable path, so one message can resolve +/// a whole screenful of icons rather than one message per icon. +pub const C2S_FS_READ: u8 = 0x4D; + /// Sync accepted or rejected: [0x40][nonce:2][sync_id:2][status:1][detail_len:2][detail:N] /// On success detail is the canonical root (UTF-8); on failure a diagnostic. pub const S2C_FS_SYNCED: u8 = 0x40; @@ -97,6 +116,13 @@ pub const S2C_FS_UPLOAD_CHUNK: u8 = 0x4A; /// hash on `CONFLICT` (the precondition re-verified at FINISH failed). pub const S2C_FS_UPLOAD_FINISH: u8 = 0x4B; +/// Read result: [0x48][nonce:2][status:1][count:2][records:LZ4] where the +/// decompressed payload is repeated{ [status:1][path_len:2][path:N][size:4][data:size] }, +/// in the order the request asked for. Request status uses the common registry +/// (`FS_DONE_*`); each record carries its own `FS_FILE_*`, so one unreadable +/// path does not spoil the answer for the rest. +pub const S2C_FS_READ: u8 = 0x48; + /// `S2C_HELLO` feature bit: server supports the `FS_*` message family, /// reads and writes alike (docs/design/fs-watch.md, docs/design/fs-write.md). /// A read-only deployment (`BLIT_FS_WRITE=0`) still advertises this bit and @@ -210,11 +236,71 @@ pub const FS_INDEX_TRUNCATED: u8 = 1 << 0; /// small frame (the decompression guard bounds bytes, not record counts). pub const FS_INDEX_MAX_COUNT: usize = 1_000_000; -// S2C_FS_FILE status. +// S2C_FS_FILE status, shared by each `S2C_FS_READ` record. pub const FS_FILE_OK: u8 = 0; pub const FS_FILE_NOT_FOUND: u8 = 1; pub const FS_FILE_UNREADABLE: u8 = 2; pub const FS_FILE_OTHER: u8 = 3; +/// The file exists and was not read: it is over the request's `max_bytes`, or +/// over what was left of the response budget. Either way the caller has the +/// answer it needs — this path is not going to arrive — and can look elsewhere. +pub const FS_FILE_TOO_LARGE: u8 = 4; + +// C2S_FS_READ flags (docs/design/fs-read.md). + +/// Answer each group with the first path in it that can be read. +/// +/// This is the search-path question — the first of these that exists, in my +/// order of preference — which is otherwise a round trip per candidate. A path +/// that is missing, unreadable or too large is stepped over rather than +/// answered. There is exactly one record per group, in group order: a group that +/// matched nothing carries `FS_FILE_NOT_FOUND` and an empty path, so a caller +/// can align answers with questions by position. +pub const FS_READ_FIRST: u8 = 1 << 0; + +/// Answer which path, not what is in it. +/// +/// The resolution without the transfer: a caller that only needs to know *where* +/// something is — because it will hand the path to whoever actually wants the +/// bytes — pays a stat instead of a read. Records carry their status and path +/// with an empty body, and `max_bytes` still applies, so "exists but too big to +/// be what I am looking for" is still answered as such. +pub const FS_READ_NO_CONTENT: u8 = 1 << 1; + +/// The flags `FS_READ` understands; anything else answers `INVALID`. +pub const FS_READ_FLAGS_KNOWN: u8 = FS_READ_FIRST | FS_READ_NO_CONTENT; + +/// Paths one `FS_READ` may name. Generous because the shape it replaces is a +/// batch: a panel asking for a screenful of artwork, a supervisor reading every +/// `.desktop` file in a directory. +pub const FS_READ_MAX_PATHS: usize = 512; +/// Per-file ceiling when the request asks for none. +pub const FS_READ_DEFAULT_BYTES: u32 = 1024 * 1024; +/// Bytes of file content one reply carries. Whatever does not fit is reported +/// `FS_FILE_TOO_LARGE` rather than silently dropped, so a caller can re-ask for +/// the remainder in smaller batches. +pub const FS_READ_MAX_TOTAL_BYTES: usize = 8 * 1024 * 1024; + +// C2S_FS_INDEX flags. + +/// List directories instead of files. +/// +/// The shape of a tree without its contents, which is what a search path is: +/// an icon theme is fifty directories holding fifty thousand files, and a +/// caller that wants somewhere to look should not be handed all of them. +pub const FS_INDEX_DIRS_ONLY: u8 = 1 << 0; + +/// Descend through symbolic links to directories. +/// +/// Off by default, because a source tree's links can point anywhere and a walk +/// that follows them is a walk with no bound the caller chose. It exists because +/// some trees are *made* of links: on a Nix system every directory under +/// `/run/current-system/sw/share/icons` is one, so a walk that stops at them +/// reports a theme's name and nothing inside it. +pub const FS_INDEX_FOLLOW_LINKS: u8 = 1 << 1; + +/// The flags `FS_INDEX` understands; anything else answers `INVALID`. +pub const FS_INDEX_FLAGS_KNOWN: u8 = FS_INDEX_DIRS_ONLY | FS_INDEX_FOLLOW_LINKS; // FS_DONE status — the common registry (docs/protocol.md "Common status // registry"), NOT FS_SYNCED's grandfathered 0-4. @@ -991,6 +1077,171 @@ pub fn parse_fs_index_result(data: &[u8]) -> Option<(u16, u8, u8, Vec)> Some((nonce, status, flags, paths)) } +/// Build a `C2S_FS_READ`. `max_bytes` of zero asks for the server default. +/// +/// Paths come in groups, and a group is one question. Without `FS_READ_FIRST` +/// the groups are read straight through and the distinction does not matter; with +/// it each group is answered by its own first readable path, which is what lets +/// one message resolve a screenful of icons instead of one message per icon. +pub fn msg_fs_read(nonce: u16, flags: u8, max_bytes: u32, groups: &[&[&str]]) -> Option> { + let total: usize = groups.iter().map(|group| group.len()).sum(); + if groups.is_empty() || total == 0 || total > FS_READ_MAX_PATHS { + return None; + } + let group_count = u16::try_from(groups.len()).ok()?; + let mut m = Vec::with_capacity(10 + total * 32); + m.push(C2S_FS_READ); + m.extend_from_slice(&nonce.to_le_bytes()); + m.push(flags); + m.extend_from_slice(&max_bytes.to_le_bytes()); + m.extend_from_slice(&group_count.to_le_bytes()); + for group in groups { + let count = u16::try_from(group.len()).ok()?; + m.extend_from_slice(&count.to_le_bytes()); + for path in *group { + if u16::try_from(path.len()).is_err() { + return None; + } + push_str(&mut m, path); + } + } + Some(m) +} + +/// Build a one-group `C2S_FS_READ`, the plain "read these files" case. +pub fn msg_fs_read_paths(nonce: u16, flags: u8, max_bytes: u32, paths: &[&str]) -> Option> { + msg_fs_read(nonce, flags, max_bytes, &[paths]) +} + +/// Parse a `C2S_FS_READ` → `(nonce, flags, max_bytes, groups)`. +pub fn parse_fs_read(data: &[u8]) -> Option<(u16, u8, u32, Vec>)> { + // [0x4D][nonce:2][flags:1][max_bytes:4][group_count:2] + // then group_count × ( [path_count:2] then path_count × [len:2][path:N] ) + if data.first().copied() != Some(C2S_FS_READ) || data.len() < 10 { + return None; + } + let nonce = u16::from_le_bytes([data[1], data[2]]); + let flags = data[3]; + let max_bytes = u32::from_le_bytes([data[4], data[5], data[6], data[7]]); + let group_count = u16::from_le_bytes([data[8], data[9]]) as usize; + if group_count == 0 || group_count > FS_READ_MAX_PATHS { + return None; + } + let mut groups = Vec::with_capacity(group_count); + let mut total = 0usize; + let mut off = 10; + for _ in 0..group_count { + if off + 2 > data.len() { + return None; + } + let count = u16::from_le_bytes([data[off], data[off + 1]]) as usize; + off += 2; + total = total.checked_add(count)?; + if total > FS_READ_MAX_PATHS { + return None; + } + let mut paths = Vec::with_capacity(count); + for _ in 0..count { + if off + 2 > data.len() { + return None; + } + let len = u16::from_le_bytes([data[off], data[off + 1]]) as usize; + off += 2; + if off + len > data.len() { + return None; + } + paths.push(String::from_utf8(data[off..off + len].to_vec()).ok()?); + off += len; + } + groups.push(paths); + } + if off != data.len() || total == 0 { + return None; + } + Some((nonce, flags, max_bytes, groups)) +} + +/// Build an `S2C_FS_READ` from `(status, path, content)` records, in request +/// order. A record whose status is not `FS_FILE_OK` carries no content. +pub fn msg_fs_read_result(nonce: u16, status: u8, records: &[(u8, &str, &[u8])]) -> Vec { + let mut raw = Vec::with_capacity( + records + .iter() + .map(|(_, path, data)| 7 + path.len() + data.len()) + .sum::(), + ); + for (record_status, path, content) in records { + raw.push(*record_status); + push_str(&mut raw, path); + let content: &[u8] = if *record_status == FS_FILE_OK { + content + } else { + &[] + }; + raw.extend_from_slice(&(content.len() as u32).to_le_bytes()); + raw.extend_from_slice(content); + } + let compressed = lz4_flex::compress_prepend_size(&raw); + let mut m = Vec::with_capacity(6 + compressed.len()); + m.push(S2C_FS_READ); + m.extend_from_slice(&nonce.to_le_bytes()); + m.push(status); + m.extend_from_slice(&(records.len() as u16).to_le_bytes()); + m.extend_from_slice(&compressed); + m +} + +/// One answered path: its `FS_FILE_*` status, the path as it was asked for, and +/// its content — empty unless the status is `FS_FILE_OK`. +pub type FsReadRecord = (u8, String, Vec); + +/// Parse an `S2C_FS_READ` → `(nonce, status, records)`. Applies the standard +/// decompression guard; `None` = malformed or a payload disagreeing with `count`. +pub fn parse_fs_read_result(data: &[u8]) -> Option<(u16, u8, Vec)> { + // [0x48][nonce:2][status:1][count:2][records:LZ4] + if data.first().copied() != Some(S2C_FS_READ) || data.len() < 6 { + return None; + } + let nonce = u16::from_le_bytes([data[1], data[2]]); + let status = data[3]; + let count = u16::from_le_bytes([data[4], data[5]]) as usize; + if count > FS_READ_MAX_PATHS { + return None; + } + let raw = decompress_guarded(&data[6..])?; + // Each record is at least seven bytes, which bounds the preallocation. + if count > raw.len() / 7 + 1 { + return None; + } + let mut records = Vec::with_capacity(count); + let mut off = 0; + while off < raw.len() { + if off + 3 > raw.len() { + return None; + } + let record_status = raw[off]; + let path_len = u16::from_le_bytes([raw[off + 1], raw[off + 2]]) as usize; + off += 3; + if off + path_len + 4 > raw.len() { + return None; + } + let path = String::from_utf8_lossy(&raw[off..off + path_len]).into_owned(); + off += path_len; + let size = + u32::from_le_bytes([raw[off], raw[off + 1], raw[off + 2], raw[off + 3]]) as usize; + off += 4; + if off + size > raw.len() { + return None; + } + records.push((record_status, path, raw[off..off + size].to_vec())); + off += size; + } + if records.len() != count { + return None; + } + Some((nonce, status, records)) +} + pub fn msg_fs_synced(nonce: u16, sync_id: u16, status: u8, detail: &str) -> Vec { let db = detail.as_bytes(); let mut msg = Vec::with_capacity(8 + db.len()); @@ -2802,4 +3053,87 @@ mod tests { let left: Vec<_> = map.keys().cloned().collect(); assert_eq!(left, vec!["ab".to_string(), "z".to_string()]); } + + #[test] + fn read_request_round_trips() { + let paths = ["/usr/share/applications/vlc.desktop", "/etc/os-release"]; + let wire = msg_fs_read_paths(7, 0, 4096, &paths).expect("builds"); + assert_eq!( + parse_fs_read(&wire), + Some(( + 7, + 0, + 4096, + vec![paths.iter().map(|p| (*p).to_string()).collect()] + )) + ); + } + + /// One group per question, which is what makes a screenful of icons one + /// message: each group is answered by its own first readable path. + #[test] + fn a_grouped_request_keeps_its_groups() { + let first: &[&str] = &["/i/scalable/apps/a.svg", "/i/48x48/apps/a.png"]; + let second: &[&str] = &["/i/scalable/apps/b.svg"]; + let wire = msg_fs_read(9, FS_READ_FIRST, 0, &[first, second]).expect("builds"); + let (nonce, flags, max_bytes, groups) = parse_fs_read(&wire).expect("parses"); + assert_eq!((nonce, flags, max_bytes), (9, FS_READ_FIRST, 0)); + assert_eq!(groups.len(), 2); + assert_eq!(groups[0].len(), 2); + assert_eq!(groups[1], vec!["/i/scalable/apps/b.svg".to_string()]); + } + + #[test] + fn read_result_carries_a_status_per_path() { + let wire = msg_fs_read_result( + 7, + FS_DONE_OK, + &[ + (FS_FILE_OK, "/etc/os-release", b"NAME=NixOS\n"), + (FS_FILE_NOT_FOUND, "/nope", b""), + // Oversized paths carry no body even if one is passed. + (FS_FILE_TOO_LARGE, "/huge.png", b"ignored"), + ], + ); + let (nonce, status, records) = parse_fs_read_result(&wire).expect("parses"); + assert_eq!((nonce, status), (7, FS_DONE_OK)); + assert_eq!( + records, + vec![ + ( + FS_FILE_OK, + "/etc/os-release".to_string(), + b"NAME=NixOS\n".to_vec() + ), + (FS_FILE_NOT_FOUND, "/nope".to_string(), Vec::new()), + (FS_FILE_TOO_LARGE, "/huge.png".to_string(), Vec::new()), + ] + ); + } + + #[test] + fn read_rejects_what_it_cannot_answer() { + assert!(msg_fs_read(1, 0, 0, &[]).is_none()); + assert!(msg_fs_read_paths(1, 0, 0, &[]).is_none()); + // The cap is on paths in total, however they are grouped. + let too_many: Vec<&str> = vec!["/x"; FS_READ_MAX_PATHS + 1]; + assert!(msg_fs_read_paths(1, 0, 0, &too_many).is_none()); + // A count that outruns the body, and trailing bytes past the last path. + assert!(parse_fs_read(&[C2S_FS_READ, 1, 0, 0, 0, 0, 0, 0, 2, 0]).is_none()); + let mut trailing = msg_fs_read_paths(1, 0, 0, &["/x"]).expect("builds"); + trailing.push(0); + assert!(parse_fs_read(&trailing).is_none()); + // Zero paths is not a request the server should have to interpret. + assert!(parse_fs_read(&[C2S_FS_READ, 1, 0, 0, 0, 0, 0, 0, 0, 0]).is_none()); + } + + #[test] + fn an_empty_read_result_is_a_valid_answer() { + // What FIRST reports when nothing matched: no records, status OK. + let wire = msg_fs_read_result(9, FS_DONE_OK, &[]); + assert_eq!( + parse_fs_read_result(&wire), + Some((9, FS_DONE_OK, Vec::new())) + ); + } } diff --git a/crates/remote/src/git.rs b/crates/remote/src/git.rs index cdf2c46a..1b06abb9 100644 --- a/crates/remote/src/git.rs +++ b/crates/remote/src/git.rs @@ -14,10 +14,15 @@ use std::collections::BTreeMap; /// `S2C_HELLO` feature bit: server supports the `GIT_*` message family. pub const FEATURE_GIT: u32 = 1 << 7; -// Opcodes: one contiguous block, `0xA0`-`0xB4`, grouped by role — +// Opcodes: one contiguous block, `0xA0`-`0xB5`, grouped by role — // lifecycle, pushed state, revision and log, object reads, then the // repository-wide operations. Request and response share a value where -// they pair. `0xB5`-`0xBF` are reserved for the family's next additions. +// they pair. `0xB6`-`0xBF` are reserved for the family's next additions. +// +// The block is contiguous because the server routes the whole family by +// range (`C2S_GIT_OPEN..=` the last opcode). Extending it means moving that +// bound too: an opcode outside the range is dropped before any handler runs, +// which reads as a request that never gets a reply. // C2S opcodes. @@ -71,6 +76,11 @@ pub const C2S_GIT_BLAME: u8 = 0xB2; pub const C2S_GIT_REFLOG: u8 = 0xB3; /// Fetch from a remote: [0xB4][nonce:2][repo_id:2][flags:1][timeout_ms:4][remote_len:2][remote:N][n_refspecs:2][(len:2, refspec:N)·N] pub const C2S_GIT_FETCH: u8 = 0xB4; +/// List the repository's worktrees: [0xB5][nonce:2][repo_id:2][flags:1][after_pos:8] +/// The main worktree plus every linked one, whether or not the repo was +/// opened through a linked worktree. Allocates no repo ids: naming the +/// other worktrees is not opening them. +pub const C2S_GIT_WORKTREES: u8 = 0xB5; // S2C opcodes. @@ -115,6 +125,8 @@ pub const S2C_GIT_BLAME: u8 = 0xB2; pub const S2C_GIT_REFLOG: u8 = 0xB3; /// Fetch response: [0xB4][nonce:2][status:1][flags:1][records:LZ4] pub const S2C_GIT_FETCH: u8 = 0xB4; +/// Worktree list response: [0xB5][nonce:2][status:1][flags:1][records:LZ4] +pub const S2C_GIT_WORKTREES: u8 = 0xB5; // Common status registry: every `status` byte in the family // (docs/protocol.md "Common status registry"). `FS_SYNCED` has a distinct, @@ -317,6 +329,7 @@ pub const GIT_PATCH_TRUNCATED: u8 = 1 << 1; pub const GIT_DISCOVER_TRUNCATED: u8 = 1 << 0; pub const GIT_BLAME_TRUNCATED: u8 = 1 << 0; pub const GIT_REFLOG_TRUNCATED: u8 = 1 << 0; +pub const GIT_WORKTREES_TRUNCATED: u8 = 1 << 0; // C2S_GIT_DISCOVER request flags. /// Descend into a repository once one is found (off by default, so a tree @@ -365,6 +378,8 @@ pub const GIT_STATE_RECORD_STATUS: u8 = 0x04; pub const GIT_STATE_RECORD_UPSTREAM: u8 = 0x05; pub const GIT_STATE_RECORD_STASH: u8 = 0x06; pub const GIT_STATE_RECORD_REMOTE: u8 = 0x07; +/// The worktree set's generation (see [`GitStateRecord::WorktreeGen`]). +pub const GIT_STATE_RECORD_WORKTREE_GEN: u8 = 0x08; // STATE_REMOTE record flags. /// The remote whose `HEAD` the checkout's default branch tracks. @@ -440,11 +455,33 @@ pub const GIT_PATCH_FILE_FILTERED: u8 = 1 << 1; // Record kind inside the GIT_INDEX response. pub const GIT_INDEX_RECORD_ENTRY: u8 = 0x04; -// Record kinds inside the 0xB1-0xB4 responses. +// Record kinds inside the 0xB1-0xB5 responses. pub const GIT_DISCOVER_RECORD_REPO: u8 = 0x01; pub const GIT_BLAME_RECORD_RANGE: u8 = 0x01; pub const GIT_REFLOG_RECORD_ENTRY: u8 = 0x01; pub const GIT_FETCH_RECORD_REF: u8 = 0x01; +pub const GIT_WORKTREES_RECORD_TREE: u8 = 0x01; + +// WORKTREE record flags. +/// The main worktree — the one whose `.git` is a directory. Exactly one +/// record carries this, including when the repo was opened through a +/// linked worktree. +pub const GIT_WORKTREE_MAIN: u8 = 1 << 0; +/// The worktree this `repo_id` was opened at, so a client can mark where +/// it already is instead of comparing paths it may have canonicalized +/// differently. +pub const GIT_WORKTREE_CURRENT: u8 = 1 << 1; +/// `git worktree lock`ed; `lock_reason` carries why, when one was given. +pub const GIT_WORKTREE_LOCKED: u8 = 1 << 2; +/// The worktree directory is gone from disk (`git worktree prune` would +/// drop the administrative entry). Reported rather than hidden: a stale +/// entry is the thing a client most needs to be told about. +pub const GIT_WORKTREE_PRUNABLE: u8 = 1 << 3; +/// HEAD is detached, so `branch` is empty. +pub const GIT_WORKTREE_DETACHED: u8 = 1 << 4; +/// Bare: no worktree directory at all, so `path` is empty. Only ever the +/// main record. +pub const GIT_WORKTREE_BARE: u8 = 1 << 5; // REPO_FOUND record flags. pub const GIT_FOUND_BARE: u8 = 1 << 0; @@ -1365,6 +1402,46 @@ pub fn parse_git_fetch(msg: &[u8]) -> Option> { }) } +/// A decoded `C2S_GIT_WORKTREES`. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct GitWorktreesRequest { + pub nonce: u16, + pub repo_id: u16, + /// No flags are defined yet; a non-zero value is `INVALID` rather than + /// ignored, so a future bit cannot be silently dropped by an old server. + pub flags: u8, + /// Worktrees already delivered, so a repository with more of them than + /// the entry budget pages: re-issue with the `CURSOR`'s `pos`. Like a + /// reflog, the enumeration has no name to resume from — the + /// administrative directory is read in sorted order — so the position + /// is the key. + pub after_pos: u64, +} + +pub fn msg_git_worktrees(req: &GitWorktreesRequest) -> Vec { + let mut msg = Vec::with_capacity(14); + msg.push(C2S_GIT_WORKTREES); + msg.extend_from_slice(&req.nonce.to_le_bytes()); + msg.extend_from_slice(&req.repo_id.to_le_bytes()); + msg.push(req.flags); + msg.extend_from_slice(&req.after_pos.to_le_bytes()); + msg +} + +pub fn parse_git_worktrees(msg: &[u8]) -> Option { + let mut b = body_of(msg, C2S_GIT_WORKTREES)?; + let nonce = take_u16(&mut b)?; + let repo_id = take_u16(&mut b)?; + let flags = take_u8(&mut b)?; + let after_pos = take_u64(&mut b)?; + Some(GitWorktreesRequest { + nonce, + repo_id, + flags, + after_pos, + }) +} + // --------------------------------------------------------------------------- // S2C message builders and parsers // --------------------------------------------------------------------------- @@ -1834,6 +1911,24 @@ pub enum GitStateRecord<'a> { fetch_url: &'a str, push_url: &'a str, }, + /// WORKTREE_GEN 0x08: [kind:1][count:4][digest:8] + /// + /// Names the current worktree *set* without describing it: `count` is + /// how many worktrees exist (the main one included) and `digest` is + /// order-independent over their administrative identities and lock + /// state. A client refetches [`C2S_GIT_WORKTREES`] when either moves. + /// + /// It is in the state stream rather than the list being pushed there + /// because resolving one worktree's HEAD costs opening its gitdir — + /// unaffordable on every ref settle — while this is a directory read. + /// And it has to be in the snapshot at all, not merely watched, or + /// identical-snapshot suppression would swallow the push: adding or + /// removing a worktree leaves every ref and status record byte for + /// byte the same. + /// + /// Emitted once per snapshot, always, so a `0`/`0` record is "no + /// worktrees dir" rather than "an old server that never said". + WorktreeGen { count: u32, digest: u64 }, } /// Append one record to an uncompressed `GIT_STATE` records buffer. @@ -1922,6 +2017,11 @@ pub fn append_git_state_record(buf: &mut Vec, record: &GitStateRecord<'_>) { push_str(buf, fetch_url); push_str(buf, push_url); } + GitStateRecord::WorktreeGen { count, digest } => { + buf.push(GIT_STATE_RECORD_WORKTREE_GEN); + buf.extend_from_slice(&count.to_le_bytes()); + buf.extend_from_slice(&digest.to_le_bytes()); + } } end_record(buf, start); } @@ -2024,6 +2124,11 @@ impl<'a> Iterator for GitStateRecordIter<'a> { push_url, }); } + GIT_STATE_RECORD_WORKTREE_GEN => { + let count = take_u32(&mut b)?; + let digest = take_u64(&mut b)?; + return Some(GitStateRecord::WorktreeGen { count, digest }); + } _ => continue, // unknown kind: skip via record_len } } @@ -2686,6 +2791,11 @@ records_resp!( records_resp!(msg_git_blame_resp, parse_git_blame_resp, S2C_GIT_BLAME); records_resp!(msg_git_reflog_resp, parse_git_reflog_resp, S2C_GIT_REFLOG); records_resp!(msg_git_fetch_resp, parse_git_fetch_resp, S2C_GIT_FETCH); +records_resp!( + msg_git_worktrees_resp, + parse_git_worktrees_resp, + S2C_GIT_WORKTREES +); /// One decoded record from a `GIT_DISCOVER` response payload. #[derive(Clone, Debug, PartialEq, Eq)] @@ -3008,6 +3118,93 @@ impl<'a> Iterator for GitFetchRecordIter<'a> { } } +/// One decoded record from a `GIT_WORKTREES` response payload. +#[derive(Clone, Debug, PartialEq, Eq)] +pub enum GitWorktreeRecord<'a> { + /// TREE 0x01: [kind:1][flags:1][oid:32][path_len:2][path:N][branch_len:2][branch:N][lock_len:2][lock_reason:N] + Tree { + flags: u8, + /// The commit that worktree's HEAD resolves to; zero when unborn, + /// or when the worktree is `PRUNABLE` and its HEAD is unreadable. + oid: GitOid, + /// Escaped canonical worktree root; empty when `BARE`. Escaped + /// rather than plain UTF-8 because it is a path read back off + /// disk, not one the client chose. + path: &'a str, + /// Full ref name HEAD points at (`refs/heads/…`); empty when + /// `DETACHED`. + branch: &'a str, + /// Why it is locked; empty unless `LOCKED`, and empty even then + /// when the lock was taken with no reason. + lock_reason: &'a str, + }, + Cursor { + after: &'a str, + pos: u64, + }, +} + +pub fn append_git_worktree_record(buf: &mut Vec, record: &GitWorktreeRecord<'_>) { + match record { + GitWorktreeRecord::Cursor { after, pos } => push_cursor_record(buf, after, *pos), + GitWorktreeRecord::Tree { + flags, + oid, + path, + branch, + lock_reason, + } => { + let start = begin_record(buf); + buf.push(GIT_WORKTREES_RECORD_TREE); + buf.push(*flags); + buf.extend_from_slice(oid); + push_str(buf, path); + push_str(buf, branch); + push_str(buf, lock_reason); + end_record(buf, start); + } + } +} + +pub struct GitWorktreeRecordIter<'a> { + data: &'a [u8], +} + +pub fn git_worktree_records(data: &[u8]) -> GitWorktreeRecordIter<'_> { + GitWorktreeRecordIter { data } +} + +impl<'a> Iterator for GitWorktreeRecordIter<'a> { + type Item = GitWorktreeRecord<'a>; + + fn next(&mut self) -> Option> { + loop { + let (kind, mut b) = next_record(&mut self.data)?; + match kind { + GIT_WORKTREES_RECORD_TREE => { + let flags = take_u8(&mut b)?; + let oid = take_oid(&mut b)?; + let path = take_str(&mut b)?; + let branch = take_str(&mut b)?; + let lock_reason = take_str(&mut b)?; + return Some(GitWorktreeRecord::Tree { + flags, + oid, + path, + branch, + lock_reason, + }); + } + GIT_RECORD_CURSOR => { + let (after, pos) = take_cursor(&mut b)?; + return Some(GitWorktreeRecord::Cursor { after, pos }); + } + _ => continue, + } + } + } +} + // --------------------------------------------------------------------------- // Client-side state reducer // --------------------------------------------------------------------------- @@ -3102,6 +3299,12 @@ pub struct GitStateMirror { pub stashes: Vec, /// Keyed by remote name; populated with the `REMOTES` open flag. pub remotes: BTreeMap, + /// The worktree set's `(count, digest)`. Refetch `GIT_WORKTREES` + /// whenever this changes; `(0, 0)` is a bare repository with no linked + /// worktrees (and also what an old server that never sends the record + /// leaves behind, which is why the panel treats a first sighting as a + /// change rather than waiting for one). + pub worktree_gen: (u32, u64), /// The last snapshot's truncation flags (`GIT_STATE_*_TRUNCATED`). pub flags: u8, /// Records accumulated from `PARTIAL` chunks of the snapshot in @@ -3278,6 +3481,9 @@ impl GitStateMirror { }, ); } + GitStateRecord::WorktreeGen { count, digest } => { + next.worktree_gen = (count, digest); + } } } *self = next; @@ -4731,3 +4937,111 @@ mod tests { assert_eq!(git_index_records(&index).count(), 1); } } + +#[cfg(test)] +mod worktree_wire_tests { + use super::*; + + /// The bytes pinned in `js/core/src/__tests__/git.test.ts`. Both sides + /// assert against this same hex, so an encoder change on either one has + /// to be a deliberate edit in two languages rather than a silent skew. + const PINNED_WORKTREES: &str = concat!( + // TREE: len 62, kind 01, flags 03 (MAIN|CURRENT), oid 0x11×20 + // zero-padded to 32, "/w/main", "refs/heads/main", empty lock. + "3e000000", + "0103", + "1111111111111111111111111111111111111111", + "000000000000000000000000", + "07002f772f6d61696e", + "0f00726566732f68656164732f6d61696e", + "0000", + // TREE: len 51, flags 14 (LOCKED|DETACHED), oid 0x22×20, "/w/wt", + // empty branch (detached), lock reason "on usb". + "33000000", + "0114", + "2222222222222222222222222222222222222222", + "000000000000000000000000", + "05002f772f7774", + "0000", + "06006f6e20757362", + ); + + fn hex(bytes: &[u8]) -> String { + bytes.iter().map(|b| format!("{b:02x}")).collect() + } + + fn oid_of(byte: u8) -> GitOid { + let mut oid = GIT_OID_NONE; + oid[..20].fill(byte); + oid + } + + #[test] + fn worktree_records_match_the_pinned_bytes() { + let mut buf = Vec::new(); + append_git_worktree_record( + &mut buf, + &GitWorktreeRecord::Tree { + flags: GIT_WORKTREE_MAIN | GIT_WORKTREE_CURRENT, + oid: oid_of(0x11), + path: "/w/main", + branch: "refs/heads/main", + lock_reason: "", + }, + ); + append_git_worktree_record( + &mut buf, + &GitWorktreeRecord::Tree { + flags: GIT_WORKTREE_LOCKED | GIT_WORKTREE_DETACHED, + oid: oid_of(0x22), + path: "/w/wt", + branch: "", + lock_reason: "on usb", + }, + ); + assert_eq!(hex(&buf), PINNED_WORKTREES.replace(['\n', ' '], "")); + } + + /// Round-trip, including the family-wide `CURSOR` a truncated page ends + /// with. + #[test] + fn worktree_records_round_trip() { + let mut buf = Vec::new(); + let tree = GitWorktreeRecord::Tree { + flags: GIT_WORKTREE_PRUNABLE, + oid: GIT_OID_NONE, + path: "/gone", + branch: "refs/heads/x", + lock_reason: "", + }; + let cursor = GitWorktreeRecord::Cursor { + after: "", + pos: 256, + }; + append_git_worktree_record(&mut buf, &tree); + append_git_worktree_record(&mut buf, &cursor); + let got: Vec<_> = git_worktree_records(&buf).collect(); + assert_eq!(got, vec![tree, cursor]); + } + + /// The generation record, and that an unknown record kind between two + /// known ones is skipped by `record_len` rather than derailing the + /// iterator — the forward-compatibility the family relies on. + #[test] + fn worktree_gen_survives_an_unknown_neighbour() { + let mut buf = Vec::new(); + let generation = GitStateRecord::WorktreeGen { + count: 4, + digest: 0xdead_beef_0000_0001, + }; + append_git_state_record(&mut buf, &generation); + // A record of a kind this build has never heard of. + let start = begin_record(&mut buf); + buf.push(0x7E); + buf.extend_from_slice(b"from the future"); + end_record(&mut buf, start); + append_git_state_record(&mut buf, &generation); + let got: Vec<_> = git_state_records(&buf).collect(); + assert_eq!(got, vec![generation.clone(), generation]); + } +} diff --git a/crates/remote/src/journal.rs b/crates/remote/src/journal.rs new file mode 100644 index 00000000..e5ea7c2a --- /dev/null +++ b/crates/remote/src/journal.rs @@ -0,0 +1,723 @@ +//! Per-command terminal journal and sequence-addressed output +//! (docs/design/term-journal.md). +//! +//! A shell that emits OSC 133 semantic-prompt markers tells the server where +//! each command begins, where its output starts, and how it ended. The server +//! turns that into a bounded ring of [`CommandRecord`]s per PTY and hands a +//! client the record list, one command's output, or everything appended since +//! a cursor — the three questions an agent driving a long-lived shell actually +//! asks. +//! +//! Output is addressed by **sequence**, not by grid coordinate. A sequence is +//! the absolute index of a grid line since the PTY was created +//! (`scrolled_lines + row`), so it stays valid as scrollback evicts underneath +//! it: an evicted region is detectable (`oldest_seq`) rather than silently +//! misread. `(seq, col)` together form a byte-exact cursor. +//! +//! All integers little-endian, tightly packed, as everywhere in the protocol. + +/// List journal records: [0x50][nonce:2][pty_id:2][from_index:8][limit:2][flags:1] +/// `from_index` is the first record wanted; with [`JOURNAL_TAIL`] it instead +/// counts back from the newest, so `from_index = 0` means "the last `limit`". +pub const C2S_TERM_JOURNAL: u8 = 0x50; +/// Fetch one command's output: [0x51][nonce:2][pty_id:2][index:8][max_bytes:4][flags:1] +/// `index` = [`JOURNAL_INDEX_LATEST`] selects the newest record. +pub const C2S_TERM_OUTPUT: u8 = 0x51; +/// Read everything appended since a cursor: +/// [0x52][nonce:2][pty_id:2][from_seq:8][from_col:2][max_bytes:4][flags:1] +/// Answered with `S2C_TERM_OUTPUT`, whose `next_seq`/`next_col` are the +/// cursor to send back next time. +pub const C2S_TERM_SINCE: u8 = 0x52; +/// Block until a command finishes: [0x53][nonce:2][pty_id:2][index:8][timeout_ms:4] +/// `index` = [`JOURNAL_INDEX_LATEST`] waits on whatever is running now, or on +/// the next command to start if the shell is at a prompt. Exactly one +/// `S2C_TERM_COMMAND` comes back per nonce; on timeout it carries the record +/// as it stands, still flagged [`RECORD_RUNNING`]. +pub const C2S_TERM_JOURNAL_WAIT: u8 = 0x53; + +/// Journal listing: [0x50][nonce:2][pty_id:2][status:1][oldest_index:8][next_index:8][count:2][records…] +/// `oldest_index` is the lowest index still retained (records below it were +/// evicted by the ring bound); `next_index` is the index the next command +/// will take, so `next_index - oldest_index` is what the journal holds. +pub const S2C_TERM_JOURNAL: u8 = 0x50; +/// Output: [0x51][nonce:2][pty_id:2][status:1][flags:1][start_seq:8][start_col:2][next_seq:8][next_col:2][text_len:4][text:N] +/// Answers both `C2S_TERM_OUTPUT` and `C2S_TERM_SINCE`; `start_*` is where +/// the returned text actually begins after clamping, `next_*` is the cursor +/// to resume from. +pub const S2C_TERM_OUTPUT: u8 = 0x51; +/// One command record: [0x52][nonce:2][pty_id:2][status:1][record] +/// Answer to `C2S_TERM_JOURNAL_WAIT`. +pub const S2C_TERM_COMMAND: u8 = 0x52; + +/// `S2C_HELLO` feature bit: the server keeps a per-command journal driven by +/// OSC 133 and serves sequence-addressed output (docs/design/term-journal.md). +/// `BLIT_TERM_JOURNAL=0` withholds the bit *and* refuses every nonce-bearing +/// request with `PERMISSION`, so a client that ignores feature bits still +/// gets its one reply. +pub const FEATURE_TERM_JOURNAL: u32 = 1 << 28; + +/// `from_index` counts back from the newest record rather than up from the +/// oldest. `C2S_TERM_JOURNAL` flags bit 0. +pub const JOURNAL_TAIL: u8 = 1 << 0; + +/// `C2S_TERM_SINCE` flags bit 0: report the current cursor and return no +/// text. This is how a client establishes a starting cursor without first +/// pulling everything already on screen. +pub const SINCE_PROBE: u8 = 1 << 0; + +/// `index` sentinel meaning "the newest record" (`C2S_TERM_OUTPUT`) or "the +/// running command, else the next one to start" (`C2S_TERM_JOURNAL_WAIT`). +pub const JOURNAL_INDEX_LATEST: u64 = u64::MAX; + +// S2C_TERM_OUTPUT flags. +/// `max_bytes` cut the response short. `next_seq`/`next_col` name the first +/// line not included, so the client pages forward by re-asking from there. +pub const OUTPUT_TRUNCATED: u8 = 1 << 0; +/// The requested start had already scrolled out of the scrollback; the text +/// begins at the oldest line still retained. +pub const OUTPUT_EVICTED: u8 = 1 << 1; +/// The PTY is on the alternate screen, where sequences do not advance +/// (docs/design/term-journal.md § Alternate screen). +pub const OUTPUT_ALT_SCREEN: u8 = 1 << 2; + +// CommandRecord flags. +/// The command has not finished. +pub const RECORD_RUNNING: u8 = 1 << 0; +/// `exit_code` is meaningful. Absent when the shell's `D` marker carried no +/// status, which is common in the wild. +pub const RECORD_HAS_EXIT: u8 = 1 << 1; +/// The command line could not be recovered — an `OSC 133 ; C` arrived with no +/// preceding `B` to delimit the input region. +pub const RECORD_NO_COMMAND: u8 = 1 << 2; +/// Closed by a new prompt rather than by a `D` marker: the shell was +/// interrupted, reset, or emits only a subset of the markers. +pub const RECORD_INCOMPLETE: u8 = 1 << 3; +/// The start of this command's output has been evicted from the scrollback, +/// so fetching it returns only the surviving tail. +pub const RECORD_EVICTED: u8 = 1 << 4; +/// The PTY's process exited while this command was still running. +pub const RECORD_PTY_EXITED: u8 = 1 << 5; + +/// Fixed-size head of an encoded record, before the command text. +const RECORD_HEAD: usize = 8 + 1 + 4 + 8 + 8 + 8 + 8 + 2; + +/// One command as the server observed it between OSC 133 markers. +/// +/// The output region is a half-open sequence range `[start_seq, end_seq)` of +/// absolute grid lines. While the command runs, `end_seq` is the live bottom +/// and moves; once it completes it is frozen. +#[derive(Clone, Debug, Default, PartialEq, Eq)] +pub struct CommandRecord { + /// Monotonic per PTY, stable across reads, never reused. + pub index: u64, + pub flags: u8, + /// Meaningful only under [`RECORD_HAS_EXIT`]. + pub exit_code: i32, + /// First grid line of the command's output. + pub start_seq: u64, + /// One past the last grid line of the output. + pub end_seq: u64, + /// Unix epoch milliseconds; 0 when unknown. + pub started_ms: u64, + /// Unix epoch milliseconds; 0 while running. + pub ended_ms: u64, + /// The command line, recovered from the region between the `B` and `C` + /// markers (or given verbatim by `OSC 633 ; E`). Empty under + /// [`RECORD_NO_COMMAND`]. + pub command: String, +} + +impl CommandRecord { + pub fn running(&self) -> bool { + self.flags & RECORD_RUNNING != 0 + } + + pub fn exit(&self) -> Option { + (self.flags & RECORD_HAS_EXIT != 0).then_some(self.exit_code) + } + + pub fn encode(&self, out: &mut Vec) { + let cmd = self.command.as_bytes(); + let cmd_len = cmd.len().min(u16::MAX as usize); + out.reserve(RECORD_HEAD + cmd_len); + out.extend_from_slice(&self.index.to_le_bytes()); + out.push(self.flags); + out.extend_from_slice(&self.exit_code.to_le_bytes()); + out.extend_from_slice(&self.start_seq.to_le_bytes()); + out.extend_from_slice(&self.end_seq.to_le_bytes()); + out.extend_from_slice(&self.started_ms.to_le_bytes()); + out.extend_from_slice(&self.ended_ms.to_le_bytes()); + out.extend_from_slice(&(cmd_len as u16).to_le_bytes()); + out.extend_from_slice(&cmd[..cmd_len]); + } + + /// Decode one record, returning it with the number of bytes consumed. + pub fn decode(buf: &[u8]) -> Option<(Self, usize)> { + if buf.len() < RECORD_HEAD { + return None; + } + let u64_at = |o: usize| u64::from_le_bytes(buf[o..o + 8].try_into().unwrap()); + let cmd_len = u16::from_le_bytes([buf[RECORD_HEAD - 2], buf[RECORD_HEAD - 1]]) as usize; + let end = RECORD_HEAD + cmd_len; + if buf.len() < end { + return None; + } + let record = Self { + index: u64_at(0), + flags: buf[8], + exit_code: i32::from_le_bytes(buf[9..13].try_into().unwrap()), + start_seq: u64_at(13), + end_seq: u64_at(21), + started_ms: u64_at(29), + ended_ms: u64_at(37), + command: String::from_utf8_lossy(&buf[RECORD_HEAD..end]).into_owned(), + }; + Some((record, end)) + } +} + +/// A parsed `C2S_TERM_JOURNAL`. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct JournalRequest { + pub nonce: u16, + pub pty_id: u16, + pub from_index: u64, + pub limit: u16, + pub flags: u8, +} + +/// A parsed `C2S_TERM_OUTPUT`. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct OutputRequest { + pub nonce: u16, + pub pty_id: u16, + pub index: u64, + pub max_bytes: u32, + pub flags: u8, +} + +/// A parsed `C2S_TERM_SINCE`. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct SinceRequest { + pub nonce: u16, + pub pty_id: u16, + pub from_seq: u64, + pub from_col: u16, + pub max_bytes: u32, + pub flags: u8, +} + +/// A parsed `C2S_TERM_JOURNAL_WAIT`. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct WaitRequest { + pub nonce: u16, + pub pty_id: u16, + pub index: u64, + pub timeout_ms: u32, +} + +/// A decoded `S2C_TERM_OUTPUT`. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct OutputReply { + pub nonce: u16, + pub pty_id: u16, + pub status: u8, + pub flags: u8, + pub start_seq: u64, + pub start_col: u16, + pub next_seq: u64, + pub next_col: u16, + pub text: String, +} + +/// A decoded `S2C_TERM_JOURNAL`. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct JournalReply { + pub nonce: u16, + pub pty_id: u16, + pub status: u8, + pub oldest_index: u64, + pub next_index: u64, + pub records: Vec, +} + +fn u16_at(buf: &[u8], o: usize) -> u16 { + u16::from_le_bytes([buf[o], buf[o + 1]]) +} + +fn u32_at(buf: &[u8], o: usize) -> u32 { + u32::from_le_bytes(buf[o..o + 4].try_into().unwrap()) +} + +fn u64_at(buf: &[u8], o: usize) -> u64 { + u64::from_le_bytes(buf[o..o + 8].try_into().unwrap()) +} + +pub fn msg_term_journal( + nonce: u16, + pty_id: u16, + from_index: u64, + limit: u16, + flags: u8, +) -> Vec { + let mut msg = Vec::with_capacity(16); + msg.push(C2S_TERM_JOURNAL); + msg.extend_from_slice(&nonce.to_le_bytes()); + msg.extend_from_slice(&pty_id.to_le_bytes()); + msg.extend_from_slice(&from_index.to_le_bytes()); + msg.extend_from_slice(&limit.to_le_bytes()); + msg.push(flags); + msg +} + +pub fn parse_term_journal(data: &[u8]) -> Option { + if data.len() < 16 || data[0] != C2S_TERM_JOURNAL { + return None; + } + Some(JournalRequest { + nonce: u16_at(data, 1), + pty_id: u16_at(data, 3), + from_index: u64_at(data, 5), + limit: u16_at(data, 13), + flags: data[15], + }) +} + +pub fn msg_term_output(nonce: u16, pty_id: u16, index: u64, max_bytes: u32, flags: u8) -> Vec { + let mut msg = Vec::with_capacity(18); + msg.push(C2S_TERM_OUTPUT); + msg.extend_from_slice(&nonce.to_le_bytes()); + msg.extend_from_slice(&pty_id.to_le_bytes()); + msg.extend_from_slice(&index.to_le_bytes()); + msg.extend_from_slice(&max_bytes.to_le_bytes()); + msg.push(flags); + msg +} + +pub fn parse_term_output(data: &[u8]) -> Option { + if data.len() < 18 || data[0] != C2S_TERM_OUTPUT { + return None; + } + Some(OutputRequest { + nonce: u16_at(data, 1), + pty_id: u16_at(data, 3), + index: u64_at(data, 5), + max_bytes: u32_at(data, 13), + flags: data[17], + }) +} + +pub fn msg_term_since( + nonce: u16, + pty_id: u16, + from_seq: u64, + from_col: u16, + max_bytes: u32, + flags: u8, +) -> Vec { + let mut msg = Vec::with_capacity(20); + msg.push(C2S_TERM_SINCE); + msg.extend_from_slice(&nonce.to_le_bytes()); + msg.extend_from_slice(&pty_id.to_le_bytes()); + msg.extend_from_slice(&from_seq.to_le_bytes()); + msg.extend_from_slice(&from_col.to_le_bytes()); + msg.extend_from_slice(&max_bytes.to_le_bytes()); + msg.push(flags); + msg +} + +pub fn parse_term_since(data: &[u8]) -> Option { + if data.len() < 20 || data[0] != C2S_TERM_SINCE { + return None; + } + Some(SinceRequest { + nonce: u16_at(data, 1), + pty_id: u16_at(data, 3), + from_seq: u64_at(data, 5), + from_col: u16_at(data, 13), + max_bytes: u32_at(data, 15), + flags: data[19], + }) +} + +pub fn msg_term_journal_wait(nonce: u16, pty_id: u16, index: u64, timeout_ms: u32) -> Vec { + let mut msg = Vec::with_capacity(17); + msg.push(C2S_TERM_JOURNAL_WAIT); + msg.extend_from_slice(&nonce.to_le_bytes()); + msg.extend_from_slice(&pty_id.to_le_bytes()); + msg.extend_from_slice(&index.to_le_bytes()); + msg.extend_from_slice(&timeout_ms.to_le_bytes()); + msg +} + +pub fn parse_term_journal_wait(data: &[u8]) -> Option { + if data.len() < 17 || data[0] != C2S_TERM_JOURNAL_WAIT { + return None; + } + Some(WaitRequest { + nonce: u16_at(data, 1), + pty_id: u16_at(data, 3), + index: u64_at(data, 5), + timeout_ms: u32_at(data, 13), + }) +} + +pub fn msg_s2c_term_journal( + nonce: u16, + pty_id: u16, + status: u8, + oldest_index: u64, + next_index: u64, + records: &[CommandRecord], +) -> Vec { + let count = records.len().min(u16::MAX as usize); + let mut msg = Vec::with_capacity(24 + count * RECORD_HEAD); + msg.push(S2C_TERM_JOURNAL); + msg.extend_from_slice(&nonce.to_le_bytes()); + msg.extend_from_slice(&pty_id.to_le_bytes()); + msg.push(status); + msg.extend_from_slice(&oldest_index.to_le_bytes()); + msg.extend_from_slice(&next_index.to_le_bytes()); + msg.extend_from_slice(&(count as u16).to_le_bytes()); + for record in &records[..count] { + record.encode(&mut msg); + } + msg +} + +pub fn parse_s2c_term_journal(data: &[u8]) -> Option { + if data.len() < 24 || data[0] != S2C_TERM_JOURNAL { + return None; + } + let count = u16_at(data, 22) as usize; + let mut records = Vec::with_capacity(count); + let mut at = 24; + for _ in 0..count { + let (record, used) = CommandRecord::decode(&data[at..])?; + records.push(record); + at += used; + } + Some(JournalReply { + nonce: u16_at(data, 1), + pty_id: u16_at(data, 3), + status: data[5], + oldest_index: u64_at(data, 6), + next_index: u64_at(data, 14), + records, + }) +} + +#[allow(clippy::too_many_arguments)] +pub fn msg_s2c_term_output( + nonce: u16, + pty_id: u16, + status: u8, + flags: u8, + start_seq: u64, + start_col: u16, + next_seq: u64, + next_col: u16, + text: &str, +) -> Vec { + let mut msg = Vec::with_capacity(31 + text.len()); + msg.push(S2C_TERM_OUTPUT); + msg.extend_from_slice(&nonce.to_le_bytes()); + msg.extend_from_slice(&pty_id.to_le_bytes()); + msg.push(status); + msg.push(flags); + msg.extend_from_slice(&start_seq.to_le_bytes()); + msg.extend_from_slice(&start_col.to_le_bytes()); + msg.extend_from_slice(&next_seq.to_le_bytes()); + msg.extend_from_slice(&next_col.to_le_bytes()); + msg.extend_from_slice(&(text.len() as u32).to_le_bytes()); + msg.extend_from_slice(text.as_bytes()); + msg +} + +pub fn parse_s2c_term_output(data: &[u8]) -> Option { + if data.len() < 31 || data[0] != S2C_TERM_OUTPUT { + return None; + } + let text_len = u32_at(data, 27) as usize; + if data.len() < 31 + text_len { + return None; + } + Some(OutputReply { + nonce: u16_at(data, 1), + pty_id: u16_at(data, 3), + status: data[5], + flags: data[6], + start_seq: u64_at(data, 7), + start_col: u16_at(data, 15), + next_seq: u64_at(data, 17), + next_col: u16_at(data, 25), + text: String::from_utf8_lossy(&data[31..31 + text_len]).into_owned(), + }) +} + +pub fn msg_s2c_term_command( + nonce: u16, + pty_id: u16, + status: u8, + record: &CommandRecord, +) -> Vec { + let mut msg = Vec::with_capacity(6 + RECORD_HEAD); + msg.push(S2C_TERM_COMMAND); + msg.extend_from_slice(&nonce.to_le_bytes()); + msg.extend_from_slice(&pty_id.to_le_bytes()); + msg.push(status); + record.encode(&mut msg); + msg +} + +/// Returns `(nonce, pty_id, status, record)`. +pub fn parse_s2c_term_command(data: &[u8]) -> Option<(u16, u16, u8, CommandRecord)> { + if data.len() < 6 || data[0] != S2C_TERM_COMMAND { + return None; + } + let (record, _) = CommandRecord::decode(&data[6..])?; + Some((u16_at(data, 1), u16_at(data, 3), data[5], record)) +} + +/// The nonce of any journal-family C2S message, for the blanket refusal a +/// disabled family owes every request (`BLIT_TERM_JOURNAL=0`). +pub fn journal_nonce(data: &[u8]) -> Option { + if data.len() < 5 { + return None; + } + matches!( + data[0], + C2S_TERM_JOURNAL | C2S_TERM_OUTPUT | C2S_TERM_SINCE | C2S_TERM_JOURNAL_WAIT + ) + .then(|| u16_at(data, 1)) +} + +/// The refusal a disabled family sends for `data`, shaped like the reply the +/// caller was waiting for so one nonce still gets one answer. +pub fn refusal(data: &[u8], status: u8) -> Option> { + if data.len() < 5 { + return None; + } + let nonce = u16_at(data, 1); + let pty_id = u16_at(data, 3); + match data[0] { + C2S_TERM_JOURNAL => Some(msg_s2c_term_journal(nonce, pty_id, status, 0, 0, &[])), + C2S_TERM_OUTPUT | C2S_TERM_SINCE => Some(msg_s2c_term_output( + nonce, pty_id, status, 0, 0, 0, 0, 0, "", + )), + C2S_TERM_JOURNAL_WAIT => Some(msg_s2c_term_command( + nonce, + pty_id, + status, + &CommandRecord::default(), + )), + _ => None, + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn sample() -> CommandRecord { + CommandRecord { + index: 7, + flags: RECORD_HAS_EXIT | RECORD_EVICTED, + exit_code: -1, + start_seq: 1200, + end_seq: 1234, + started_ms: 1_700_000_000_000, + ended_ms: 1_700_000_001_500, + command: "cargo test --workspace".into(), + } + } + + #[test] + fn record_round_trips() { + let record = sample(); + let mut buf = Vec::new(); + record.encode(&mut buf); + let (back, used) = CommandRecord::decode(&buf).expect("decode"); + assert_eq!(back, record); + assert_eq!(used, buf.len()); + assert_eq!(back.exit(), Some(-1)); + assert!(!back.running()); + } + + #[test] + fn record_decode_refuses_short_buffers() { + let mut buf = Vec::new(); + sample().encode(&mut buf); + for cut in 0..buf.len() { + assert!( + CommandRecord::decode(&buf[..cut]).is_none(), + "a {cut}-byte prefix must not decode" + ); + } + } + + #[test] + fn journal_request_round_trips() { + let msg = msg_term_journal(9, 3, 42, 20, JOURNAL_TAIL); + let req = parse_term_journal(&msg).expect("parse"); + assert_eq!( + req, + JournalRequest { + nonce: 9, + pty_id: 3, + from_index: 42, + limit: 20, + flags: JOURNAL_TAIL, + } + ); + } + + #[test] + fn since_request_round_trips() { + let msg = msg_term_since(1, 2, u64::MAX, 17, 65536, SINCE_PROBE); + let req = parse_term_since(&msg).expect("parse"); + assert_eq!(req.from_seq, u64::MAX); + assert_eq!(req.from_col, 17); + assert_eq!(req.max_bytes, 65536); + assert_eq!(req.flags, SINCE_PROBE); + } + + #[test] + fn output_and_wait_requests_round_trip() { + let out = msg_term_output(4, 5, JOURNAL_INDEX_LATEST, 4096, 0); + let req = parse_term_output(&out).expect("parse"); + assert_eq!(req.index, JOURNAL_INDEX_LATEST); + assert_eq!(req.max_bytes, 4096); + + let wait = msg_term_journal_wait(6, 7, 11, 30_000); + let req = parse_term_journal_wait(&wait).expect("parse"); + assert_eq!( + req, + WaitRequest { + nonce: 6, + pty_id: 7, + index: 11, + timeout_ms: 30_000, + } + ); + } + + #[test] + fn parsers_reject_the_wrong_opcode() { + let msg = msg_term_journal(1, 1, 0, 1, 0); + assert!(parse_term_output(&msg).is_none()); + assert!(parse_term_since(&msg).is_none()); + assert!(parse_term_journal_wait(&msg).is_none()); + } + + #[test] + fn journal_reply_round_trips_with_records() { + let records = vec![ + sample(), + CommandRecord { + index: 8, + flags: RECORD_RUNNING | RECORD_NO_COMMAND, + ..CommandRecord::default() + }, + ]; + let msg = msg_s2c_term_journal(3, 1, crate::STATUS_OK, 5, 9, &records); + let reply = parse_s2c_term_journal(&msg).expect("parse"); + assert_eq!(reply.nonce, 3); + assert_eq!(reply.oldest_index, 5); + assert_eq!(reply.next_index, 9); + assert_eq!(reply.records, records); + assert!(reply.records[1].running()); + assert_eq!(reply.records[1].exit(), None); + } + + #[test] + fn output_reply_round_trips() { + let msg = msg_s2c_term_output( + 2, + 4, + crate::STATUS_OK, + OUTPUT_TRUNCATED | OUTPUT_EVICTED, + 100, + 0, + 140, + 13, + "hello\nworld", + ); + let reply = parse_s2c_term_output(&msg).expect("parse"); + assert_eq!(reply.start_seq, 100); + assert_eq!(reply.next_seq, 140); + assert_eq!(reply.next_col, 13); + assert_eq!(reply.text, "hello\nworld"); + assert_eq!(reply.flags, OUTPUT_TRUNCATED | OUTPUT_EVICTED); + } + + #[test] + fn command_reply_round_trips() { + let record = sample(); + let msg = msg_s2c_term_command(5, 6, crate::STATUS_OK, &record); + let (nonce, pty_id, status, back) = parse_s2c_term_command(&msg).expect("parse"); + assert_eq!((nonce, pty_id, status), (5, 6, crate::STATUS_OK)); + assert_eq!(back, record); + } + + #[test] + fn refusals_answer_every_request_shape() { + for msg in [ + msg_term_journal(1, 2, 0, 10, 0), + msg_term_output(1, 2, 0, 10, 0), + msg_term_since(1, 2, 0, 0, 10, 0), + msg_term_journal_wait(1, 2, 0, 10), + ] { + assert_eq!(journal_nonce(&msg), Some(1)); + let reply = refusal(&msg, crate::STATUS_PERMISSION).expect("refusal"); + let status = match reply[0] { + S2C_TERM_JOURNAL => parse_s2c_term_journal(&reply).map(|r| r.status), + S2C_TERM_OUTPUT => parse_s2c_term_output(&reply).map(|r| r.status), + S2C_TERM_COMMAND => parse_s2c_term_command(&reply).map(|r| r.2), + other => panic!("unexpected refusal opcode {other:#04x}"), + }; + assert_eq!(status, Some(crate::STATUS_PERMISSION)); + } + } + + #[test] + fn journal_nonce_ignores_foreign_opcodes() { + assert_eq!(journal_nonce(&[crate::C2S_READ, 1, 0, 2, 0]), None); + assert_eq!(journal_nonce(&[C2S_TERM_JOURNAL, 1]), None); + } + + #[test] + fn feature_bit_is_free() { + // Bits 0-27 were taken when this family landed; the mask below is + // every bit the tree defines, and the new one must not collide. + let taken = crate::FEATURE_CREATE_NONCE + | crate::FEATURE_RESTART + | crate::FEATURE_RESIZE_BATCH + | crate::FEATURE_COPY_RANGE + | crate::FEATURE_COMPOSITOR + | crate::FEATURE_AUDIO + | crate::fs::FEATURE_FS + | crate::git::FEATURE_GIT + | crate::lsp::FEATURE_LSP + | crate::kv::FEATURE_KV + | crate::net::FEATURE_NET + | crate::extension::FEATURE_EXTENSION + | crate::channel::FEATURE_CHANNEL + | crate::process::FEATURE_PROCESS + | crate::FEATURE_CREATE_STATUS + | crate::FEATURE_KILL_MODE + | crate::FEATURE_PTY_DEADLINE + | crate::FEATURE_SCROLL_BY + | crate::FEATURE_SURFACE_TOUCH + | crate::FEATURE_SURFACE_TEXT_INPUT + | crate::FEATURE_CLIENT_CONTROL + | crate::desktop::FEATURE_DESKTOP + | crate::media::FEATURE_DESKTOP_MEDIA + | crate::process::FEATURE_PROCESS_SESSION_ENV + | crate::env::FEATURE_ENV + | crate::process::FEATURE_APP_SOCKET + | crate::channel::FEATURE_CHANNEL_WATCH + | crate::FEATURE_CLIENT_ORIGIN; + assert_eq!(taken & FEATURE_TERM_JOURNAL, 0); + } +} diff --git a/crates/remote/src/lib.rs b/crates/remote/src/lib.rs index 95c9df6e..2ff5b144 100644 --- a/crates/remote/src/lib.rs +++ b/crates/remote/src/lib.rs @@ -19,6 +19,10 @@ pub mod lsp; /// client-side mirror reducer. pub mod kv; +/// The server's own environment (docs/design/env.md): the only way a client +/// learns anything about the session it is attached to. +pub mod env; + /// TCP and UDP relay (docs/design/net.md): opcodes, flags, and the /// message builders both ends share. pub mod net; @@ -35,6 +39,11 @@ pub mod extension; /// Viewer media input, compositor portals, and MPRIS state/control. pub mod media; +/// Per-command terminal journal and sequence-addressed output +/// (docs/design/term-journal.md): opcodes, the command record codec, and the +/// cursor a client feeds back to read only what is new. +pub mod journal; + /// Cap on any single LZ4-decompressed payload, protocol-wide /// (docs/protocol.md "Compressed payloads"). Receivers check the prepended /// size against it *before* allocating, so a hostile or corrupt length @@ -149,7 +158,14 @@ pub const C2S_CLIENT_METRICS: u8 = 0x05; /// (but its arrival resets any server-side receive timeout). pub const C2S_PING: u8 = 0x08; /// Enumerate every other connection to this server: -/// [0x09][nonce:2]. The server replies with [`S2C_CLIENT_LIST`]. +/// [0x09][nonce:2], or [0x09][nonce:2][flags:1]. The server replies with +/// [`S2C_CLIENT_LIST`], or with [`S2C_CLIENT_LIST2`] when the flags byte +/// carries [`CLIENT_LIST_WANT_ORIGIN`]. +/// +/// The flags byte is only safe to send once [`FEATURE_CLIENT_ORIGIN`] is +/// advertised: every server answers a client-control request with unexpected +/// trailing bytes with `INVALID`, so an unconditional flag would make the +/// catalog unreadable on an older server rather than degrade. pub const C2S_CLIENT_LIST: u8 = 0x09; /// Disconnect another connection: /// [0x0A][nonce:2][client_id:8][reason:N]. @@ -158,9 +174,30 @@ pub const C2S_CLIENT_LIST: u8 = 0x09; /// [`S2C_KICKED`] carrying the UTF-8 reason and the server closes its /// connection. A client may not kick itself. pub const C2S_KICK: u8 = 0x0A; -/// Subscribe to live connection-catalog snapshots: [0x0B][nonce:2]. -/// Every update uses [`S2C_CLIENT_LIST`] with the same nonce. +/// Ask for a Wayland socket dedicated to one application: +/// [0x0D][nonce:2][app_len:2][app_id][inst_len:2][instance_id]. +/// +/// The server binds a fresh socket, tells the compositor that every client +/// arriving on it belongs to `(app_id, instance_id)`, and replies with +/// [`S2C_APP_SOCKET`] carrying the basename to put in `WAYLAND_DISPLAY`. +/// Because the socket is bound before the reply is sent, the caller may spawn +/// the application the moment it arrives. +/// +/// This is how identity becomes trustworthy: the application never gets to say +/// who it is, so nothing it does can change the answer. Pair it with +/// `PROCESS_SPAWN_SESSION_ENV` for the rest of the session environment and an +/// explicit `WAYLAND_DISPLAY` entry, which wins over the session's own. +pub const C2S_APP_SOCKET: u8 = 0x0D; +/// Subscribe to live connection-catalog snapshots: [0x0B][nonce:2], or +/// [0x0B][nonce:2][flags:1] under the same rule as [`C2S_CLIENT_LIST`]. +/// Every update uses [`S2C_CLIENT_LIST`] — or [`S2C_CLIENT_LIST2`] when the +/// watch asked for origins — with the same nonce. The shape a watch is opened +/// with is the shape all of its updates keep. pub const C2S_CLIENT_WATCH: u8 = 0x0B; +/// Bit 0 of the optional flags byte on [`C2S_CLIENT_LIST`] and +/// [`C2S_CLIENT_WATCH`]: answer with [`S2C_CLIENT_LIST2`], whose entries say +/// where each connection came from. +pub const CLIENT_LIST_WANT_ORIGIN: u8 = 1 << 0; /// Stop a live connection-catalog subscription: [0x0C][nonce:2]. pub const C2S_CLIENT_UNWATCH: u8 = 0x0C; /// Mouse event: [0x06][pty_id:2][type:1][button:1][col:2][row:2] @@ -588,6 +625,34 @@ pub const S2C_CLIENT_LIST: u8 = 0x12; /// terminal/surface/subscription records. Named because the encoder, the /// parser and both of the parser's bounds checks have to agree on it. pub const CLIENT_LIST_ENTRY_HEADER: usize = 38; +/// [`S2C_CLIENT_LIST`] with an origin block appended to every entry: +/// [0x15][nonce:2][self_id:8][count:4][client:N]... +/// +/// Each entry is byte-for-byte the [`S2C_CLIENT_LIST`] entry followed by +/// `[origin_kind:1][origin_len:2][origin:origin_len]`. `origin_kind` is one of +/// the `CLIENT_ORIGIN_*` values and `origin_len` is what a reader skips when +/// the kind means nothing to it — which is what lets a later kind carry a +/// payload of its own without a third opcode. +/// +/// A separate opcode rather than a flag inside `S2C_CLIENT_LIST`, because the +/// shipped parsers on both sides reject a catalog with bytes left over: the +/// wider entry has to be unreadable to them by construction, not by +/// convention. Sent only in answer to a request that set +/// [`CLIENT_LIST_WANT_ORIGIN`]. +pub const S2C_CLIENT_LIST2: u8 = 0x15; +/// The connection is an ordinary client of the server: a browser, a CLI, a +/// forwarder. `origin_len` is zero — nothing further is known about it, and +/// nothing further is claimed. +pub const CLIENT_ORIGIN_NETWORK: u8 = 0; +/// The connection belongs to a running extension attempt, and its origin +/// payload is +/// `[extension_id:8][definition_revision:8][attempt:8][task_id:4][name:N]`. +/// +/// `name` is the durable name of a persistent definition, or the label a +/// transient `ext run` was started under, and is empty when the attempt has +/// neither. It is captured when the connection is opened, so it names the +/// definition this attempt actually started from. +pub const CLIENT_ORIGIN_EXTENSION: u8 = 1; /// Correlated kick outcome: /// [0x13][nonce:2][status:1][detail:N]. /// @@ -662,6 +727,14 @@ pub const S2C_SURFACE_TITLE: u8 = 0x23; pub const S2C_SURFACE_RESIZED: u8 = 0x24; /// A Wayland surface's app_id changed: [0x28][surface_id:2][app_id:N] pub const S2C_SURFACE_APP_ID: u8 = 0x28; +/// Stamped application identity for a surface, as opposed to the +/// self-asserted `S2C_SURFACE_APP_ID`: +/// [0x32][surface_id:2][engine_len:2][engine][app_len:2][app][inst_len:2][inst] +pub const S2C_SURFACE_ORIGIN: u8 = 0x32; +/// Answer to [`C2S_APP_SOCKET`]: [0x33][nonce:2][status:1][name_len:2][name]. +/// `name` is the `WAYLAND_DISPLAY` basename, empty unless `status` is +/// [`STATUS_OK`]. +pub const S2C_APP_SOCKET: u8 = 0x33; /// A Wayland client asked for its toplevel to be activated /// (xdg_activation_v1) — raise and focus the matching pane: /// [0x2D][surface_id:2] @@ -915,6 +988,12 @@ pub const FEATURE_SURFACE_TEXT_INPUT: u32 = 1 << 19; /// Enumerating server connections and kicking another connection, including /// the correlated result and terminal `S2C_KICKED` reason. pub const FEATURE_CLIENT_CONTROL: u32 = 1 << 20; +/// `C2S_CLIENT_LIST` and `C2S_CLIENT_WATCH` accept the optional +/// [`CLIENT_LIST_WANT_ORIGIN`] flags byte, and answer it with +/// [`S2C_CLIENT_LIST2`] entries that say which connections are extension +/// attempts. Implies [`FEATURE_CLIENT_CONTROL`]. +pub const FEATURE_CLIENT_ORIGIN: u32 = 1 << 27; +// Bit 28 is [`journal::FEATURE_TERM_JOURNAL`], with the rest of the family. #[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] pub enum Color { @@ -2320,6 +2399,13 @@ pub enum ServerMsg<'a> { surface_id: u16, app_id: &'a str, }, + /// Stamped identity, as opposed to `SurfaceAppId`'s self-assertion. + SurfaceOrigin { + surface_id: u16, + sandbox_engine: &'a str, + app_id: &'a str, + instance_id: &'a str, + }, SurfaceActivated { surface_id: u16, }, @@ -2424,6 +2510,56 @@ pub struct ClientListEntry { pub terminals: Vec, pub surfaces: Vec, pub subscriptions: Vec, + /// Where the connection came from, or `None` when the catalog was the + /// plain [`S2C_CLIENT_LIST`] — which is not the same as "an ordinary + /// client", and is why this is an `Option` rather than a defaulted + /// [`ClientOrigin::Network`]. + pub origin: Option, +} + +/// The server's own account of what opened a connection, as carried by +/// [`S2C_CLIENT_LIST2`]. +#[derive(Clone, Debug, PartialEq, Eq)] +pub enum ClientOrigin { + Network, + Extension { + extension_id: u64, + definition_revision: u64, + attempt: u64, + task_id: u32, + name: String, + }, + /// A kind this build has no name for, kept whole so a reader can still say + /// "not an ordinary client" without pretending to know more. + Unknown { + kind: u8, + payload: Vec, + }, +} + +/// Interpret one `S2C_CLIENT_LIST2` origin block. +/// +/// A payload that does not fit its kind is [`ClientOrigin::Unknown`] rather +/// than a parse failure: the length prefix already told us where the entry +/// ends, so one origin this build cannot read costs that origin and nothing +/// else in the catalog. +fn parse_client_origin(kind: u8, payload: &[u8]) -> ClientOrigin { + match kind { + CLIENT_ORIGIN_NETWORK => ClientOrigin::Network, + CLIENT_ORIGIN_EXTENSION if payload.len() >= 28 => ClientOrigin::Extension { + extension_id: u64::from_le_bytes(payload[0..8].try_into().expect("checked length")), + definition_revision: u64::from_le_bytes( + payload[8..16].try_into().expect("checked length"), + ), + attempt: u64::from_le_bytes(payload[16..24].try_into().expect("checked length")), + task_id: u32::from_le_bytes(payload[24..28].try_into().expect("checked length")), + name: String::from_utf8_lossy(&payload[28..]).into_owned(), + }, + _ => ClientOrigin::Unknown { + kind, + payload: payload.to_vec(), + }, + } } #[derive(Clone, Copy, Debug, PartialEq, Eq)] @@ -2443,6 +2579,18 @@ pub struct SearchResultEntry<'a> { pub context: &'a [u8], } +/// Take one `[len:2][utf8:len]` field, advancing `input` past it. +/// +/// Returns `None` on truncation or invalid UTF-8, so a malformed field fails the +/// whole parse rather than silently reading as empty. +fn take_len_prefixed_str<'a>(input: &mut &'a [u8]) -> Option<&'a str> { + let (len, tail) = input.split_at_checked(2)?; + let len = u16::from_le_bytes([len[0], len[1]]) as usize; + let (value, tail) = tail.split_at_checked(len)?; + *input = tail; + std::str::from_utf8(value).ok() +} + pub fn parse_server_msg(data: &[u8]) -> Option> { if data.is_empty() { return None; @@ -2736,6 +2884,21 @@ pub fn parse_server_msg(data: &[u8]) -> Option> { app_id, }) } + S2C_SURFACE_ORIGIN => { + let mut rest = data.get(3..)?; + let sandbox_engine = take_len_prefixed_str(&mut rest)?; + let app_id = take_len_prefixed_str(&mut rest)?; + let instance_id = take_len_prefixed_str(&mut rest)?; + if !rest.is_empty() { + return None; + } + Some(ServerMsg::SurfaceOrigin { + surface_id: u16::from_le_bytes([data[1], data[2]]), + sandbox_engine, + app_id, + instance_id, + }) + } S2C_SURFACE_ACTIVATED => { if data.len() < 3 { return None; @@ -2899,7 +3062,8 @@ pub fn parse_server_msg(data: &[u8]) -> Option> { wayland: data[1] != 0, }) } - S2C_CLIENT_LIST => { + S2C_CLIENT_LIST | S2C_CLIENT_LIST2 => { + let with_origin = data[0] == S2C_CLIENT_LIST2; if data.len() < 15 { return None; } @@ -2969,6 +3133,18 @@ pub fn parse_server_msg(data: &[u8]) -> Option> { }); offset += 3; } + let origin = if with_origin { + let kind = *data.get(offset)?; + let length = + u16::from_le_bytes(data.get(offset + 1..offset + 3)?.try_into().ok()?) + as usize; + let end = offset.checked_add(3)?.checked_add(length)?; + let payload = data.get(offset + 3..end)?; + offset = end; + Some(parse_client_origin(kind, payload)) + } else { + None + }; clients.push(ClientListEntry { client_id, age_secs, @@ -2977,6 +3153,7 @@ pub fn parse_server_msg(data: &[u8]) -> Option> { terminals, surfaces, subscriptions, + origin, }); } if offset != data.len() { @@ -3453,6 +3630,19 @@ pub fn msg_client_watch(nonce: u16) -> Vec { msg } +/// Ask for the catalog, or start a watch, with every entry's origin — only +/// valid against a server advertising [`FEATURE_CLIENT_ORIGIN`]. +/// +/// `opcode` is [`C2S_CLIENT_LIST`] or [`C2S_CLIENT_WATCH`]. Unwatching needs no +/// flag: it names a nonce, and the shape was settled when the watch opened. +pub fn msg_client_list_with_origin(opcode: u8, nonce: u16) -> Vec { + let mut msg = Vec::with_capacity(4); + msg.push(opcode); + msg.extend_from_slice(&nonce.to_le_bytes()); + msg.push(CLIENT_LIST_WANT_ORIGIN); + msg +} + /// Stop the connection-catalog stream under `nonce`. pub fn msg_client_unwatch(nonce: u16) -> Vec { let mut msg = Vec::with_capacity(3); @@ -3490,10 +3680,30 @@ pub fn msg_kick(nonce: u16, client_id: u64, reason: &str) -> Vec { /// Reply to [`C2S_CLIENT_LIST`]. Active subscriptions with no size report use /// zeroes for their size fields. +/// +/// Any [`ClientListEntry::origin`] is dropped: this shape has nowhere to put +/// it, which is exactly what makes it readable by a client that never asked. pub fn msg_s2c_client_list(nonce: u16, self_id: u64, clients: &[ClientListEntry]) -> Vec { + encode_client_list(S2C_CLIENT_LIST, nonce, self_id, clients) +} + +/// Reply to a [`C2S_CLIENT_LIST`] or [`C2S_CLIENT_WATCH`] that set +/// [`CLIENT_LIST_WANT_ORIGIN`]. An entry whose `origin` is `None` is encoded +/// as [`CLIENT_ORIGIN_NETWORK`]: the requester asked, so every entry answers. +pub fn msg_s2c_client_list2(nonce: u16, self_id: u64, clients: &[ClientListEntry]) -> Vec { + encode_client_list(S2C_CLIENT_LIST2, nonce, self_id, clients) +} + +fn encode_client_list( + opcode: u8, + nonce: u16, + self_id: u64, + clients: &[ClientListEntry], +) -> Vec { + let with_origin = opcode == S2C_CLIENT_LIST2; let count = clients.len().min(u32::MAX as usize); let mut msg = Vec::new(); - msg.push(S2C_CLIENT_LIST); + msg.push(opcode); msg.extend_from_slice(&nonce.to_le_bytes()); msg.extend_from_slice(&self_id.to_le_bytes()); msg.extend_from_slice(&(count as u32).to_le_bytes()); @@ -3523,10 +3733,47 @@ pub fn msg_s2c_client_list(nonce: u16, self_id: u64, clients: &[ClientListEntry] msg.push(subscription.kind); msg.extend_from_slice(&subscription.id.to_le_bytes()); } + if with_origin { + encode_client_origin(&mut msg, client.origin.as_ref()); + } } msg } +fn encode_client_origin(msg: &mut Vec, origin: Option<&ClientOrigin>) { + let (kind, payload) = match origin { + None | Some(ClientOrigin::Network) => (CLIENT_ORIGIN_NETWORK, Vec::new()), + Some(ClientOrigin::Extension { + extension_id, + definition_revision, + attempt, + task_id, + name, + }) => { + let name = name.as_bytes(); + let mut payload = Vec::with_capacity(28 + name.len()); + payload.extend_from_slice(&extension_id.to_le_bytes()); + payload.extend_from_slice(&definition_revision.to_le_bytes()); + payload.extend_from_slice(&attempt.to_le_bytes()); + payload.extend_from_slice(&task_id.to_le_bytes()); + payload.extend_from_slice(name); + (CLIENT_ORIGIN_EXTENSION, payload) + } + Some(ClientOrigin::Unknown { kind, payload }) => (*kind, payload.clone()), + }; + // An origin too long for the length prefix would desynchronise every later + // entry, so it degrades to "ordinary client" rather than corrupt the + // catalog. Only a 64 KiB extension name gets here. + let Ok(length) = u16::try_from(payload.len()) else { + msg.push(CLIENT_ORIGIN_NETWORK); + msg.extend_from_slice(&0u16.to_le_bytes()); + return; + }; + msg.push(kind); + msg.extend_from_slice(&length.to_le_bytes()); + msg.extend_from_slice(&payload); +} + /// Report the outcome of [`C2S_KICK`] to its requester. pub fn msg_kick_result(nonce: u16, status: u8, detail: &str) -> Vec { let detail = kick_text_bytes(detail); @@ -3645,6 +3892,89 @@ pub fn msg_surface_app_id(surface_id: u16, app_id: &str) -> Vec { msg } +/// `S2C_SURFACE_ORIGIN`: who a surface's application actually is. +/// +/// Unlike `S2C_SURFACE_APP_ID`, which forwards a string the application chose +/// for itself, every field here was stamped by whoever created the Wayland +/// socket the client connected on. Sent only for surfaces on a per-app socket, +/// and only once — identity is fixed for the life of a connection. +pub fn msg_surface_origin( + surface_id: u16, + sandbox_engine: &str, + app_id: &str, + instance_id: &str, +) -> Vec { + let engine = sandbox_engine.as_bytes(); + let app = app_id.as_bytes(); + let instance = instance_id.as_bytes(); + let mut msg = Vec::with_capacity(9 + engine.len() + app.len() + instance.len()); + msg.push(S2C_SURFACE_ORIGIN); + msg.extend_from_slice(&surface_id.to_le_bytes()); + msg.extend_from_slice(&(engine.len() as u16).to_le_bytes()); + msg.extend_from_slice(engine); + msg.extend_from_slice(&(app.len() as u16).to_le_bytes()); + msg.extend_from_slice(app); + msg.extend_from_slice(&(instance.len() as u16).to_le_bytes()); + msg.extend_from_slice(instance); + msg +} + +/// Encode `C2S_APP_SOCKET`. +pub fn msg_app_socket_request(nonce: u16, app_id: &str, instance_id: &str) -> Vec { + let app = app_id.as_bytes(); + let instance = instance_id.as_bytes(); + let mut msg = Vec::with_capacity(7 + app.len() + instance.len()); + msg.push(C2S_APP_SOCKET); + msg.extend_from_slice(&nonce.to_le_bytes()); + msg.extend_from_slice(&(app.len() as u16).to_le_bytes()); + msg.extend_from_slice(app); + msg.extend_from_slice(&(instance.len() as u16).to_le_bytes()); + msg.extend_from_slice(instance); + msg +} + +/// Decode `C2S_APP_SOCKET` into `(nonce, app_id, instance_id)`. +pub fn parse_app_socket_request(data: &[u8]) -> Option<(u16, &str, &str)> { + if data.first() != Some(&C2S_APP_SOCKET) { + return None; + } + let nonce = u16::from_le_bytes([*data.get(1)?, *data.get(2)?]); + let mut rest = data.get(3..)?; + let app_id = take_len_prefixed_str(&mut rest)?; + let instance_id = take_len_prefixed_str(&mut rest)?; + if !rest.is_empty() { + return None; + } + Some((nonce, app_id, instance_id)) +} + +/// Encode `S2C_APP_SOCKET`. +pub fn msg_app_socket_reply(nonce: u16, status: u8, wayland_display: &str) -> Vec { + let name = wayland_display.as_bytes(); + let mut msg = Vec::with_capacity(6 + name.len()); + msg.push(S2C_APP_SOCKET); + msg.extend_from_slice(&nonce.to_le_bytes()); + msg.push(status); + msg.extend_from_slice(&(name.len() as u16).to_le_bytes()); + msg.extend_from_slice(name); + msg +} + +/// Decode `S2C_APP_SOCKET` into `(nonce, status, wayland_display)`. +pub fn parse_app_socket_reply(data: &[u8]) -> Option<(u16, u8, &str)> { + if data.first() != Some(&S2C_APP_SOCKET) { + return None; + } + let nonce = u16::from_le_bytes([*data.get(1)?, *data.get(2)?]); + let status = *data.get(3)?; + let mut rest = data.get(4..)?; + let name = take_len_prefixed_str(&mut rest)?; + if !rest.is_empty() { + return None; + } + Some((nonce, status, name)) +} + pub fn msg_surface_activated(surface_id: u16) -> Vec { let mut msg = Vec::with_capacity(3); msg.push(S2C_SURFACE_ACTIVATED); @@ -5533,6 +5863,16 @@ mod tests { kind: CLIENT_SUBSCRIPTION_GIT, id: 2, }], + // Offered to the shape that cannot carry it, to pin that it is + // dropped rather than smuggled onto the end of the entry where + // a shipped parser would call the whole catalog malformed. + origin: Some(ClientOrigin::Extension { + extension_id: 0x0102_0304_0506_0708, + definition_revision: 2, + attempt: 3, + task_id: 4, + name: "systemd".to_string(), + }), }], ); assert!(matches!( @@ -5563,6 +5903,7 @@ mod tests { kind: CLIENT_SUBSCRIPTION_GIT, id: 2, }], + origin: None, }] )); assert!(parse_server_msg(&list[..list.len() - 1]).is_none()); @@ -5583,6 +5924,84 @@ mod tests { )); } + /// One entry per origin kind in a single catalog, because the risk the + /// origin block carries is not reading one entry — it is reading the + /// *next* one after a variable-length tail. + #[test] + fn client_list2_carries_origins() { + fn entry(client_id: u64, origin: Option) -> ClientListEntry { + ClientListEntry { + client_id, + age_secs: 1, + outbound_bytes_per_sec: 2, + inbound_bytes_per_sec: 3, + terminals: vec![ClientTerminalSubscription { + pty_id: 4, + rows: 24, + cols: 80, + }], + surfaces: Vec::new(), + subscriptions: Vec::new(), + origin, + } + } + + let extension = ClientOrigin::Extension { + extension_id: 0x05a3_415a_2dd1_ef9b, + definition_revision: 2, + attempt: 3, + task_id: 4, + name: "systemd".to_string(), + }; + let unknown = ClientOrigin::Unknown { + kind: 200, + payload: vec![9, 9, 9], + }; + let clients = [ + entry(1, Some(ClientOrigin::Network)), + entry(2, Some(extension.clone())), + // A kind from a later server: skipped by its length prefix, and + // the entries after it still parse. + entry(3, Some(unknown.clone())), + // Never asked about, so answered as an ordinary client rather than + // left for the reader to guess. + entry(4, None), + ]; + + let msg = msg_s2c_client_list2(9, 3, &clients); + assert_eq!(msg[0], S2C_CLIENT_LIST2); + let Some(ServerMsg::ClientList { + nonce: 9, + self_id: 3, + clients: parsed, + }) = parse_server_msg(&msg) + else { + panic!("a catalog with origins did not parse"); + }; + assert_eq!( + parsed + .iter() + .map(|entry| entry.origin.clone()) + .collect::>(), + [ + Some(ClientOrigin::Network), + Some(extension), + Some(unknown), + Some(ClientOrigin::Network), + ] + ); + assert_eq!(parsed[3].client_id, 4); + assert!(parse_server_msg(&msg[..msg.len() - 1]).is_none()); + + // The two shapes differ only in the opcode and the tails, so a reader + // that asked for one must not be handed the other. + assert_ne!(msg_s2c_client_list(9, 3, &clients), msg); + assert_eq!( + msg_client_list_with_origin(C2S_CLIENT_LIST, 7), + [C2S_CLIENT_LIST, 7, 0, CLIENT_LIST_WANT_ORIGIN] + ); + } + #[test] fn kick_text_is_utf8_safe_and_bounded() { let reason = format!("{}é", "x".repeat(KICK_REASON_MAX - 1)); diff --git a/crates/remote/src/process.rs b/crates/remote/src/process.rs index b040bacb..544d153d 100644 --- a/crates/remote/src/process.rs +++ b/crates/remote/src/process.rs @@ -9,6 +9,13 @@ use crate::{STATUS_INVALID, STATUS_OK, STATUS_TOO_LARGE}; /// `S2C_HELLO` feature bit: native non-PTY child processes. pub const FEATURE_PROCESS: u32 = 1 << 13; +/// `S2C_HELLO` feature bit: `PROCESS_SPAWN_SESSION_ENV` is understood. Separate +/// from `FEATURE_PROCESS` because the flag rejection an older server produces is +/// indistinguishable from a malformed frame, so it cannot be probed. +pub const FEATURE_PROCESS_SESSION_ENV: u32 = 1 << 23; +/// `S2C_HELLO` feature bit: `C2S_APP_SOCKET` mints per-application Wayland +/// sockets, so a surface's identity can be stamped rather than self-asserted. +pub const FEATURE_APP_SOCKET: u32 = 1 << 25; // Direction-local process-family opcodes. pub const C2S_PROCESS_SPAWN: u8 = 0xC0; @@ -29,7 +36,20 @@ pub const S2C_PROCESS_WATCHED: u8 = 0xC7; pub const PROCESS_SPAWN_MERGE_STDERR: u8 = 1 << 0; pub const PROCESS_SPAWN_DETACHABLE: u8 = 1 << 1; -pub const PROCESS_SPAWN_FLAGS: u8 = PROCESS_SPAWN_MERGE_STDERR | PROCESS_SPAWN_DETACHABLE; +/// Give the child the session environment a PTY child would get — the +/// compositor socket, the desktop bus, and the audio sockets. Without it a GUI +/// child inherits the *server's* environment, which on a desktop host means the +/// host's display or none at all. Gated on `FEATURE_PROCESS_SESSION_ENV` +/// because an older server answers an unknown flag bit with the same +/// `STATUS_INVALID` it uses for a corrupt frame. +pub const PROCESS_SPAWN_SESSION_ENV: u8 = 1 << 2; +pub const PROCESS_SPAWN_FLAGS: u8 = + PROCESS_SPAWN_MERGE_STDERR | PROCESS_SPAWN_DETACHABLE | PROCESS_SPAWN_SESSION_ENV; +/// Flags a catalog entry may carry, deliberately narrower than +/// `PROCESS_SPAWN_FLAGS`: `validate_list_entry` rejects the *entire* +/// `S2C_PROCESS_LISTED` message on an unknown bit, so echoing a newer spawn +/// flag into the catalog would make every list read fail for older clients. +pub const PROCESS_CATALOG_FLAGS: u8 = PROCESS_SPAWN_MERGE_STDERR | PROCESS_SPAWN_DETACHABLE; /// Request the process's single vacant stdin-writer role while watching. pub const PROCESS_WATCH_STDIN: u8 = 1 << 0; @@ -878,7 +898,7 @@ pub fn parse_process_controlled(msg: &[u8]) -> Result, Pro fn validate_list_entry(entry: ProcessListEntry<'_>) -> Result<(), ProcessCodecError> { if entry.process_ref == 0 || !matches!(entry.state, PROCESS_STATE_RUNNING | PROCESS_STATE_EXITED) - || entry.flags & !PROCESS_SPAWN_FLAGS != 0 + || entry.flags & !PROCESS_CATALOG_FLAGS != 0 || (entry.state == PROCESS_STATE_EXITED && entry.flags & PROCESS_SPAWN_DETACHABLE == 0) || has_nul(entry.argv0) { @@ -1655,6 +1675,45 @@ mod tests { } } + /// The catalog mask must stay a strict subset of the spawn mask. Widening + /// it to match would make `validate_list_entry` echo a newer spawn flag into + /// `S2C_PROCESS_LISTED`, and an older client rejects the whole message on an + /// unknown bit — so every catalog read would fail, not just the new entry. + #[test] + fn the_catalog_flag_mask_stays_narrower_than_the_spawn_mask() { + assert_eq!(PROCESS_SPAWN_SESSION_ENV, 1 << 2); + assert_eq!(FEATURE_PROCESS_SESSION_ENV, 1 << 23); + assert_eq!(PROCESS_CATALOG_FLAGS & PROCESS_SPAWN_SESSION_ENV, 0); + assert_eq!( + PROCESS_CATALOG_FLAGS & !PROCESS_SPAWN_FLAGS, + 0, + "catalog flags must be a subset of spawn flags" + ); + + // A spawn request may carry it... (the shared fixture deliberately holds + // a NUL in argv, so give this one a clean one). + let mut request = spawn_request(); + request.argv[1] = b"ab"; + request.flags = PROCESS_SPAWN_SESSION_ENV; + let encoded = msg_process_spawn(&request).expect("session-env spawn encodes"); + assert_eq!( + parse_process_spawn(&encoded).expect("round-trips").flags, + PROCESS_SPAWN_SESSION_ENV + ); + + // ...but a catalog entry may not. + assert!(matches!( + validate_list_entry(ProcessListEntry { + process_ref: 1, + state: PROCESS_STATE_RUNNING, + flags: PROCESS_SPAWN_SESSION_ENV, + pid: 42, + argv0: b"legcord", + }), + Err(ProcessCodecError::Invalid) + )); + } + #[test] fn allocation_is_locked() { assert_eq!(FEATURE_PROCESS, 1 << 13); @@ -1793,7 +1852,10 @@ mod tests { req.env.push((b"LANG", b"other")); assert_eq!(msg_process_spawn(&req), Err(ProcessCodecError::Invalid)); req.env.pop(); - req.flags = 1 << 2; + // The lowest bit no flag has claimed yet — move this up, don't delete + // it, when a flag takes it. The point is that unknown bits are refused + // rather than ignored. + req.flags = 1 << 3; assert_eq!(msg_process_spawn(&req), Err(ProcessCodecError::Invalid)); req.flags = PROCESS_SPAWN_DETACHABLE; req.cwd_kind = PROCESS_CWD_FROM_PTY; diff --git a/crates/server/src/app_env.rs b/crates/server/src/app_env.rs index dca61ac1..9da72678 100644 --- a/crates/server/src/app_env.rs +++ b/crates/server/src/app_env.rs @@ -1,8 +1,98 @@ //! The environment every GUI application in a session inherits. //! -//! Shared by the PTYs (where a user types the app's name) and the private -//! D-Bus session (where an activated service is launched on their behalf), so -//! an app reaches the same display either way. +//! Shared by the PTYs (where a user types the app's name), the private D-Bus +//! session (where an activated service is launched on their behalf), and native +//! children spawned with `PROCESS_SPAWN_SESSION_ENV`, so an app reaches the same +//! display however it was started. + +/// The session-scoped environment a GUI child needs. +/// +/// Two halves, because the variables that must be *absent* matter as much as the +/// ones that must be present: a child that inherits the host session's `DISPLAY` +/// will happily pick X11 and come up with no window at all. `build_child_env` +/// applies `remove` by filtering as it assembles a fresh envp; `Command`-based +/// spawns apply it with `env_remove`. +#[derive(Debug, Default, Clone)] +pub struct SessionEnv { + /// Variables to set, last-write-wins against anything inherited. + pub set: Vec<(String, String)>, + /// Variables belonging to the host session, which must not leak through. + pub remove: Vec<&'static str>, +} + +/// Resolve the session environment from the handles a live session holds. +/// +/// `TERM`/`COLORTERM` are deliberately *not* here: they describe a terminal, and +/// a native pipe child does not have one. +pub fn session_env( + wayland_display: Option<&str>, + x_display: Option<&str>, + desktop_bus: Option<&str>, + pulse_server: Option<&str>, + pipewire_remote: Option<&str>, +) -> SessionEnv { + let mut env = SessionEnv::default(); + let mut set = |key: &str, value: String| env.set.push((key.to_string(), value)); + + if let Some(wd) = wayland_display { + let wd_path = std::path::Path::new(wd); + if let Some(dir) = wd_path.parent() { + let inherited = std::env::var_os("XDG_RUNTIME_DIR"); + let differs = match &inherited { + Some(current) => std::path::Path::new(current) != dir, + None => true, + }; + if differs { + set("XDG_RUNTIME_DIR", dir.to_string_lossy().into_owned()); + } + } + // WAYLAND_DISPLAY must be just the socket filename (e.g. "wayland-2"), + // not a full path. Clients resolve it under XDG_RUNTIME_DIR. + let name = wd_path + .file_name() + .map(|n| n.to_string_lossy().into_owned()) + .unwrap_or_else(|| wd.to_string()); + set("WAYLAND_DISPLAY", name); + for (key, value) in toolkit_env(x_display) { + set(key, value); + } + } + // The host's display belongs to a different session. `toolkit_env` names + // this one when a bridge is listening; with no bridge the variable has to + // go, or a toolkit offered "wayland,x11" reaches for an X server that has + // nothing to do with blit. + if x_display.is_none() { + env.remove.push("DISPLAY"); + } + // Likewise both buses: the server's own and the host desktop's each belong + // elsewhere. The compositor-scoped bus activates portals on this Wayland + // display while still satisfying apps (notably Spotify) that require a bus. + env.remove.push("DBUS_SYSTEM_BUS_ADDRESS"); + if let Some(address) = desktop_bus { + set("DBUS_SESSION_BUS_ADDRESS", address.to_string()); + } else { + env.remove.push("DBUS_SESSION_BUS_ADDRESS"); + } + if let Some(server) = pulse_server { + set("PULSE_SERVER", server.to_string()); + } else { + // No audio pipeline — point PULSE_SERVER at a path that will make + // libpulse fail immediately. Without this, libpulse falls back to + // autospawn (`pulseaudio --start`) which hangs in headless / + // container environments. Setting PULSE_SERVER explicitly also + // prevents inheriting a host PulseAudio server that would bypass + // blit's audio pipeline. + set("PULSE_SERVER", "/dev/null".to_string()); + } + // Absolute, so it works regardless of the child's XDG_RUNTIME_DIR (which + // points at the Wayland socket directory). + if let Some(remote) = pipewire_remote { + set("PIPEWIRE_REMOTE", remote.to_string()); + } else { + env.remove.push("PIPEWIRE_REMOTE"); + } + env +} /// Toolkit steering for apps on blit's compositor. /// diff --git a/crates/server/src/audio.rs b/crates/server/src/audio.rs index b4dddcfa..95716fc8 100644 --- a/crates/server/src/audio.rs +++ b/crates/server/src/audio.rs @@ -268,6 +268,15 @@ context.modules = [ { name = libpipewire-module-adapter } { name = libpipewire-module-link-factory } { name = libpipewire-module-metadata } + # Without this, `pw-top` and `pw-profiler` print *nothing* against a blit + # daemon — they need the Profiler interface this module registers, and the + # stripped config above supplies no other route to it. That makes xruns + # unobservable in production, which is the one number worth having when a + # listener reports dropouts: the graph either missed its deadlines or it + # did not, and every other explanation lives downstream of that answer. + # The cost is a registry object; the per-cycle profiling work only runs + # while a client is actually subscribed to it. + { name = libpipewire-module-profiler } { name = libpipewire-module-spa-node-factory } ] context.objects = [ diff --git a/crates/server/src/channel.rs b/crates/server/src/channel.rs index 3c02b96e..726082bd 100644 --- a/crates/server/src/channel.rs +++ b/crates/server/src/channel.rs @@ -3,7 +3,7 @@ use blit_remote::channel::{ CHANNEL_CLOSE_PEER_GONE, CHANNEL_CLOSE_PROTOCOL_VIOLATION, CHANNEL_MAX_UNCONSUMED_MESSAGES, CHANNEL_WINDOW_BYTES, ChannelRequest, msg_channel_accepted, msg_channel_ack, - msg_channel_closed, msg_channel_data, msg_channel_opened, + msg_channel_closed, msg_channel_data, msg_channel_names, msg_channel_opened, }; use blit_remote::{ STATUS_BUDGET, STATUS_CANCELLED, STATUS_CONFLICT, STATUS_INVALID, STATUS_NOT_FOUND, STATUS_OK, @@ -16,6 +16,7 @@ use std::sync::{ }; const DEFAULT_MAX_LISTEN_PER_CLIENT: usize = 64; +const DEFAULT_MAX_WATCH_PER_CLIENT: usize = 16; const DEFAULT_MAX_LISTENERS: usize = 1024; const DEFAULT_MAX_PER_CLIENT: usize = 64; const DEFAULT_MAX_CONNECTED: usize = 128; @@ -34,6 +35,18 @@ struct Listener { token: [u8; 16], } +/// One endpoint following a fixed set of names. +/// +/// `published` is the exact last packet this watch was sent, which is what +/// makes a watch quiet: the registry changes for reasons a watcher never asked +/// about — every extension attempt mints and drops a command listener — and a +/// change that leaves this watch's answer byte-identical publishes nothing. +#[derive(Clone, Debug)] +struct Watch { + names: Vec, + published: Vec, +} + /// Immutable listener identity used to fence extension command publication. #[derive(Clone, Debug, PartialEq, Eq)] pub(crate) struct ListenerSnapshot { @@ -175,6 +188,7 @@ impl Drop for DrainReservation { #[derive(Clone, Copy, Debug)] struct Limits { listen_per_client: usize, + watch_per_client: usize, listeners: usize, handles_per_client: usize, connected_pairs: usize, @@ -185,6 +199,7 @@ impl Default for Limits { fn default() -> Self { Self { listen_per_client: DEFAULT_MAX_LISTEN_PER_CLIENT, + watch_per_client: DEFAULT_MAX_WATCH_PER_CLIENT, listeners: DEFAULT_MAX_LISTENERS, handles_per_client: DEFAULT_MAX_PER_CLIENT, connected_pairs: DEFAULT_MAX_CONNECTED, @@ -201,6 +216,10 @@ impl Limits { "BLIT_CHANNEL_MAX_LISTEN_PER_CLIENT", defaults.listen_per_client, ), + watch_per_client: crate::deployment_usize( + "BLIT_CHANNEL_MAX_WATCH_PER_CLIENT", + defaults.watch_per_client, + ), listeners: crate::deployment_usize("BLIT_CHANNEL_MAX_LISTENERS", defaults.listeners), handles_per_client: crate::deployment_usize( "BLIT_CHANNEL_MAX_PER_CLIENT", @@ -260,6 +279,13 @@ pub(crate) struct ChannelFabric { listener_names: HashMap, listeners: HashMap, handles: HashMap, + watches: HashMap, + /// Bumped by every change to `listener_names`. Watches are answered from + /// the registry, and the registry only moves when a name is claimed or + /// released — comparing this against `published_revision` keeps a busy + /// channel's DATA and ACK traffic from rebuilding every watcher's answer. + names_revision: u64, + published_revision: u64, /// Connected-handle slots include live handles and terminal handles whose /// already-emitted frames have not drained from their endpoint writer. endpoint_slots: Arc>>, @@ -282,6 +308,9 @@ impl ChannelFabric { listener_names: HashMap::new(), listeners: HashMap::new(), handles: HashMap::new(), + watches: HashMap::new(), + names_revision: 0, + published_revision: 0, endpoint_slots: Arc::new(Mutex::new(HashMap::new())), draining: Arc::new(Mutex::new(HashSet::new())), active_pairs: Arc::new(AtomicUsize::new(0)), @@ -356,6 +385,20 @@ impl ChannelFabric { endpoint: u64, request: ChannelRequest<'_>, reserve_outbox: &mut impl FnMut(u64, usize) -> Option, + ) -> Vec { + let mut deliveries = self.apply(endpoint, request, reserve_outbox); + // Every request that claims or releases a name passes through here, so + // this is the one place watches are answered from — including watches + // held by endpoints the requester has never heard of. + deliveries.extend(self.publish_names(reserve_outbox)); + deliveries + } + + fn apply( + &mut self, + endpoint: u64, + request: ChannelRequest<'_>, + reserve_outbox: &mut impl FnMut(u64, usize) -> Option, ) -> Vec { match request { ChannelRequest::Listen { @@ -384,6 +427,16 @@ impl ChannelFabric { ChannelRequest::Close { channel_id, reason } => { self.close(endpoint, channel_id, reason) } + ChannelRequest::Watch { channel_id, names } => { + self.watch(endpoint, channel_id, &names, reserve_outbox) + } + ChannelRequest::Unwatch { channel_id } => { + self.watches.remove(&ChannelKey { + endpoint, + channel_id, + }); + Vec::new() + } } } @@ -404,7 +457,9 @@ impl ChannelFabric { pub(crate) fn refuse(&self, endpoint: u64, kind: u8, channel_id: u32) -> Vec { if matches!( kind, - blit_remote::channel::CHANNEL_LISTEN | blit_remote::channel::CHANNEL_CONNECT + blit_remote::channel::CHANNEL_LISTEN + | blit_remote::channel::CHANNEL_CONNECT + | blit_remote::channel::CHANNEL_WATCH ) { vec![self.failed_open( endpoint, @@ -430,7 +485,9 @@ impl ChannelFabric { ) -> Vec { if matches!( kind, - blit_remote::channel::CHANNEL_LISTEN | blit_remote::channel::CHANNEL_CONNECT + blit_remote::channel::CHANNEL_LISTEN + | blit_remote::channel::CHANNEL_CONNECT + | blit_remote::channel::CHANNEL_WATCH ) { vec![self.failed_open(endpoint, channel_id, STATUS_INVALID, detail)] } else { @@ -497,6 +554,7 @@ impl ChannelFabric { delivery.outbox_reservation = Some(outbox_reservation); self.next_listener_generation = generation; + self.names_revision = self.names_revision.wrapping_add(1); self.listener_names.insert(name.to_owned(), key); self.listeners.insert( key, @@ -509,6 +567,117 @@ impl ChannelFabric { vec![delivery] } + /// Start following a name set, answering with the state it has right now. + /// + /// The immediate answer is the whole reason a watch needs no separate + /// acknowledgement: a client that receives it knows both that the watch + /// exists and what the registry holds, which is exactly what a + /// connect-and-close probe used to tell it once. + fn watch( + &mut self, + endpoint: u64, + channel_id: u32, + names: &[&str], + reserve_outbox: &mut impl FnMut(u64, usize) -> Option, + ) -> Vec { + let key = ChannelKey { + endpoint, + channel_id, + }; + if self.key_is_live(key) { + return vec![self.failed_open( + endpoint, + channel_id, + STATUS_CONFLICT, + "channel id is already live", + )]; + } + if self.watch_count(endpoint) >= self.limits.watch_per_client { + return vec![self.failed_open( + endpoint, + channel_id, + STATUS_BUDGET, + "channel watch budget exhausted", + )]; + } + let present: Vec<&str> = names + .iter() + .copied() + .filter(|name| self.listener_names.contains_key(*name)) + .collect(); + let packet = msg_channel_names(channel_id, &present) + .expect("validated watch names obey fixed limits"); + let Some(outbox_reservation) = reserve_outbox(endpoint, packet.len()) else { + // Hard outbox admission cancelled the endpoint; nothing was + // published and the client-created ID stays free. + return Vec::new(); + }; + self.watches.insert( + key, + Watch { + names: names.iter().map(|name| (*name).to_owned()).collect(), + published: packet.clone(), + }, + ); + vec![Delivery { + endpoint, + packet, + _reservation: None, + _slot: None, + _drain: None, + outbox_reservation: Some(outbox_reservation), + }] + } + + /// Re-answer every watch whose set of claimed names moved. + /// + /// A watch that cannot be given its packet keeps the one it last published, + /// so the next registry change tries again rather than leaving the watcher + /// permanently behind — and an endpoint whose outbox refuses a packet this + /// small is already being cancelled. + fn publish_names( + &mut self, + reserve_outbox: &mut impl FnMut(u64, usize) -> Option, + ) -> Vec { + if self.names_revision == self.published_revision || self.watches.is_empty() { + self.published_revision = self.names_revision; + return Vec::new(); + } + self.published_revision = self.names_revision; + let keys: Vec = self.watches.keys().copied().collect(); + let mut deliveries = Vec::new(); + for key in keys { + let watch = self.watches.get(&key).expect("key came from the map"); + let present: Vec<&str> = watch + .names + .iter() + .filter(|name| self.listener_names.contains_key(name.as_str())) + .map(String::as_str) + .collect(); + let packet = msg_channel_names(key.channel_id, &present) + .expect("validated watch names obey fixed limits"); + if watch.published == packet { + continue; + } + let Some(outbox_reservation) = reserve_outbox(key.endpoint, packet.len()) else { + continue; + }; + self.watches + .get_mut(&key) + .expect("key came from the map") + .published = packet.clone(); + deliveries.push(Delivery { + endpoint: key.endpoint, + packet, + _reservation: None, + _slot: None, + _drain: None, + outbox_reservation: Some(outbox_reservation), + }); + } + deliveries + } + fn connect( &mut self, endpoint: u64, @@ -816,9 +985,20 @@ impl ChannelFabric { } /// Remove every object owned by a departing endpoint and notify surviving - /// channel peers. Already accepted pairs are independent of listeners. - pub(crate) fn close_endpoint(&mut self, endpoint: u64) -> Vec { + /// channel peers and watchers. Already accepted pairs are independent of + /// listeners. + /// + /// This is where an extension's names are actually released: disabling or + /// removing a definition cancels its attempt, and the attempt's endpoint + /// closes here a beat later. A watcher therefore learns that a name is + /// gone from the teardown, never from the control call that started it. + pub(crate) fn close_endpoint_reserved( + &mut self, + endpoint: u64, + mut reserve_outbox: impl FnMut(u64, usize) -> Option, + ) -> Vec { self.peer_names.remove(&endpoint); + self.watches.retain(|key, _| key.endpoint != endpoint); let listener_keys: Vec<_> = self .listeners .keys() @@ -859,9 +1039,17 @@ impl ChannelFabric { } } } + deliveries.extend(self.publish_names(&mut reserve_outbox)); deliveries } + #[cfg(test)] + fn close_endpoint(&mut self, endpoint: u64) -> Vec { + self.close_endpoint_reserved(endpoint, |_, bytes| { + Some(OutboxReservation::untracked(bytes)) + }) + } + fn close_pair( &mut self, key: ChannelKey, @@ -911,6 +1099,7 @@ impl ChannelFabric { fn remove_listener(&mut self, key: ChannelKey) { if let Some(listener) = self.listeners.remove(&key) { self.listener_names.remove(&listener.name); + self.names_revision = self.names_revision.wrapping_add(1); } } @@ -927,6 +1116,13 @@ impl ChannelFabric { .count() } + fn watch_count(&self, endpoint: u64) -> usize { + self.watches + .keys() + .filter(|key| key.endpoint == endpoint) + .count() + } + fn reserve_pair( &self, connector_endpoint: u64, @@ -1029,6 +1225,7 @@ impl ChannelFabric { fn key_is_live(&self, key: ChannelKey) -> bool { self.listeners.contains_key(&key) || self.handles.contains_key(&key) + || self.watches.contains_key(&key) || self .draining .lock() @@ -1131,7 +1328,7 @@ mod tests { use super::*; use blit_remote::channel::{ CHANNEL_CLOSE_CANCELLED, ChannelMessage, msg_channel_connect, msg_channel_listen, - parse_channel_message, + msg_channel_unwatch, msg_channel_watch, parse_channel_message, }; fn fabric() -> ChannelFabric { @@ -1768,4 +1965,121 @@ mod tests { assert!(fabric.close_endpoint(1).is_empty()); assert!(fabric.listeners.is_empty()); } + + /// Every name state a watcher can be told about, in one life: what is + /// already claimed when it asks, a claim by a stranger, and a release the + /// watcher never sees a request for. + #[test] + fn watch_answers_now_and_on_every_later_change() { + let mut fabric = fabric(); + listen(&mut fabric, 1, 2, "blit.session.v1"); + + let opened = fabric.handle_packet( + 9, + &msg_channel_watch(2, &["blit.session.v1", "blit.systemd.v1"]).unwrap(), + ); + assert_eq!(watched_names(&opened[0]), vec!["blit.session.v1"]); + + let appeared = + fabric.handle_packet(3, &msg_channel_listen(2, "blit.systemd.v1", b"").unwrap()); + assert_eq!(appeared.len(), 2, "the listener's OPENED and the watch"); + assert_eq!(appeared[1].endpoint, 9); + assert_eq!( + watched_names(&appeared[1]), + vec!["blit.session.v1", "blit.systemd.v1"] + ); + + // The extension goes away the only way one does: its endpoint closes. + let departed = fabric.close_endpoint(3); + assert_eq!(departed.len(), 1); + assert_eq!(watched_names(&departed[0]), vec!["blit.session.v1"]); + } + + #[test] + fn watch_is_silent_about_names_it_did_not_declare() { + let mut fabric = fabric(); + let opened = fabric.handle_packet(9, &msg_channel_watch(2, &["blit.session.v1"]).unwrap()); + assert_eq!(watched_names(&opened[0]), Vec::<&str>::new()); + + // The command listeners every extension attempt mints look exactly + // like this, and a watcher must not hear a packet for each one. + let unrelated = + fabric.handle_packet(3, &msg_channel_listen(2, "blit.cli.7.1", b"").unwrap()); + assert_eq!(unrelated.len(), 1, "only the listener's own OPENED"); + assert_eq!(unrelated[0].endpoint, 3); + + let declared = + fabric.handle_packet(4, &msg_channel_listen(2, "blit.session.v1", b"").unwrap()); + assert_eq!(declared.len(), 2); + assert_eq!(watched_names(&declared[1]), vec!["blit.session.v1"]); + + // And the unrelated listener leaving is just as quiet as its arrival. + assert!(fabric.close_endpoint(3).is_empty()); + } + + #[test] + fn unwatch_stops_the_publishing_and_frees_the_id() { + let mut fabric = fabric(); + fabric.handle_packet(9, &msg_channel_watch(2, &["blit.session.v1"]).unwrap()); + assert!(fabric.key_is_live(ChannelKey { + endpoint: 9, + channel_id: 2, + })); + + assert!( + fabric + .handle_packet(9, &msg_channel_unwatch(2).unwrap()) + .is_empty(), + "an unwatch draws no reply" + ); + assert!(!fabric.key_is_live(ChannelKey { + endpoint: 9, + channel_id: 2, + })); + + let claimed = + fabric.handle_packet(1, &msg_channel_listen(2, "blit.session.v1", b"").unwrap()); + assert_eq!(claimed.len(), 1, "nothing is watching that name"); + + // The freed ID is usable as an ordinary channel, which is why a watch + // has to occupy the same ID space as one. + listen(&mut fabric, 9, 2, "watcher.turned.listener"); + } + + #[test] + fn a_watch_cannot_take_a_live_id_or_outgrow_its_budget() { + let mut fabric = fabric(); + listen(&mut fabric, 9, 2, "already.here"); + let conflict = + fabric.handle_packet(9, &msg_channel_watch(2, &["blit.session.v1"]).unwrap()); + assert!(matches!( + parse_channel_message(&conflict[0].packet).unwrap(), + Some(ChannelMessage::Opened { + status: STATUS_CONFLICT, + .. + }) + )); + + fabric.limits.watch_per_client = 1; + fabric.handle_packet(8, &msg_channel_watch(2, &["blit.session.v1"]).unwrap()); + let exhausted = + fabric.handle_packet(8, &msg_channel_watch(4, &["blit.systemd.v1"]).unwrap()); + assert!(matches!( + parse_channel_message(&exhausted[0].packet).unwrap(), + Some(ChannelMessage::Opened { + status: STATUS_BUDGET, + .. + }) + )); + assert_eq!(fabric.watch_count(8), 1); + } + + fn watched_names(delivery: &Delivery) -> Vec<&str> { + let Some(ChannelMessage::Names { names, .. }) = + parse_channel_message(&delivery.packet).unwrap() + else { + panic!("delivery is not a NAMES packet") + }; + names + } } diff --git a/crates/server/src/extension/mod.rs b/crates/server/src/extension/mod.rs index 884706a9..a04a58fa 100644 --- a/crates/server/src/extension/mod.rs +++ b/crates/server/src/extension/mod.rs @@ -5272,6 +5272,7 @@ fn origin_identity(origin: &super::ConnectionOrigin) -> Option<(u64, u64, u64, u definition_revision, attempt, task_id, + .. } = origin else { return None; @@ -6136,7 +6137,16 @@ mod tests { saw_put, "upload acknowledgement must precede automatic start" ); - assert_eq!(exit.reason, EXT_EXIT_RETURNED); + // The detail is the whole diagnosis and the bare + // comparison threw it away: this has failed in CI as + // `left: 5, right: 0` — a protocol violation — which + // says nothing about which of the two ways to reach + // one it took, and it does not reproduce locally. + assert_eq!( + exit.reason, EXT_EXIT_RETURNED, + "guest exited {} instead of returning: {}", + exit.reason, exit.detail + ); saw_exit = true; } _ => {} diff --git a/crates/server/src/extension_catalog.rs b/crates/server/src/extension_catalog.rs index c015eebc..9e2ab7aa 100644 --- a/crates/server/src/extension_catalog.rs +++ b/crates/server/src/extension_catalog.rs @@ -86,9 +86,17 @@ impl ExtensionCatalog { }); }; if let Some(parent) = path.parent() { + // Narrow permissions belong to directories we bring into being. + // A path the operator chose may sit in one we do not own -- point + // BLIT_EXTENSION_PATH at /tmp/x.redb and chmod'ing the parent + // fails with EPERM, which used to disable the whole extension + // subsystem over a directory nobody asked us to secure. + let existed = parent.is_dir(); std::fs::create_dir_all(parent) .map_err(|error| CatalogError::Storage(error.to_string()))?; - set_owner_directory(parent)?; + if !existed { + set_owner_directory(parent)?; + } } let db = redb::Database::create(&path) .map_err(|error| CatalogError::Storage(error.to_string()))?; diff --git a/crates/server/src/journal.rs b/crates/server/src/journal.rs new file mode 100644 index 00000000..3ef30a1a --- /dev/null +++ b/crates/server/src/journal.rs @@ -0,0 +1,1120 @@ +//! Per-command terminal journal driven by OSC 133 semantic-prompt markers +//! (docs/design/term-journal.md). +//! +//! A shell with shell integration enabled brackets each command with four +//! markers: `A` when the prompt starts, `B` when the prompt ends and typing +//! begins, `C` just before the command execs, and `D` when it finishes, +//! optionally carrying the exit status. This module turns that stream into a +//! bounded ring of records, each naming its output as a range of sequences +//! ([`blit_alacritty::TerminalDriver::seq_text`]) rather than of grid rows, +//! so a record stays readable until the scrollback actually evicts it. +//! +//! Shells emit subsets of the markers, emit them in the wrong order after an +//! interrupt, and sometimes emit none at all. Every transition below is +//! therefore total: there is no input sequence that leaves the machine stuck +//! or that loses a record, and a terminal whose shell emits nothing keeps an +//! empty journal at no cost. + +use std::collections::VecDeque; +use std::sync::OnceLock; +use std::time::{SystemTime, UNIX_EPOCH}; + +use blit_remote::journal::{ + CommandRecord, RECORD_EVICTED, RECORD_HAS_EXIT, RECORD_INCOMPLETE, RECORD_NO_COMMAND, + RECORD_PTY_EXITED, RECORD_RUNNING, +}; + +/// Which semantic-prompt dialect a marker was written in. A shell that emits +/// both would otherwise be counted twice, so the journal latches onto the +/// first one it sees (§ Dialects). +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Dialect { + /// `OSC 133` — the FinalTerm sequence kitty, WezTerm, foot and others + /// implement. + Osc133, + /// `OSC 633` — VS Code's superset, which additionally names the command + /// line outright instead of leaving it to be read off the grid. + Osc633, +} + +/// One semantic-prompt marker, and where in the byte stream it ended. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct SemanticMark { + pub kind: MarkKind, + pub dialect: Dialect, + /// Offset one past the marker's terminator in the scanned buffer. The + /// bytes before it must be fed to the terminal before the marker is + /// applied, or the cursor it is measured against is the wrong one. + pub at: usize, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub enum MarkKind { + /// `A` — a new prompt begins. + PromptStart, + /// `B` — the prompt has been drawn; what follows is what the user types. + InputStart, + /// `C` — the command is about to run; what follows is its output. + OutputStart, + /// `D` — the command finished, with an exit status if the shell sent one. + Finished(Option), + /// `633;E` — the command line, given verbatim rather than left to be read + /// back off the grid. + CommandLine(String), +} + +/// Longest OSC payload the scanner will hold across a chunk boundary. +/// +/// Markers are a few bytes; this only has to cover one split across two PTY +/// reads. A cap is required rather than merely tidy: without one, a stream +/// that opens an OSC and never terminates it would grow this buffer without +/// bound. +pub const CARRY_MAX: usize = 512; + +/// Longest command line retained per record. +pub fn command_max() -> usize { + static CACHED: OnceLock = OnceLock::new(); + *CACHED.get_or_init(|| env_usize("BLIT_TERM_JOURNAL_CMD_MAX", 4096)) +} + +/// Server ceiling on a client's `max_bytes`, whatever it asked for. +pub fn output_max() -> usize { + static CACHED: OnceLock = OnceLock::new(); + *CACHED.get_or_init(|| env_usize("BLIT_TERM_OUTPUT_MAX", 1 << 20).clamp(4096, 8 << 20)) +} + +/// `BLIT_TERM_JOURNAL=0` turns the family off: the feature bit is withheld, +/// every nonce-bearing request is refused with `PERMISSION`, and no PTY byte +/// is ever scanned for a marker. +/// +/// Cached, because the answer is consulted once per OSC on the PTY output +/// path and cannot change within a process. +pub fn enabled() -> bool { + static CACHED: OnceLock = OnceLock::new(); + *CACHED.get_or_init(|| !std::env::var("BLIT_TERM_JOURNAL").is_ok_and(|v| v == "0")) +} + +fn env_usize(key: &str, default: usize) -> usize { + std::env::var(key) + .ok() + .and_then(|v| v.parse().ok()) + .unwrap_or(default) +} + +fn now_ms() -> u64 { + SystemTime::now() + .duration_since(UNIX_EPOCH) + .map(|d| d.as_millis() as u64) + .unwrap_or(0) +} + +/// Parse one OSC payload into a marker, if it is one. +/// +/// `payload` is everything between `ESC ]` and the terminator. Anything that +/// is not a semantic-prompt marker — including the `133;P`, `133;L` and +/// `633;P` variants nothing here consumes — returns `None` and is left for +/// the terminal emulator, which drops it as it always has. +pub fn parse_mark(payload: &[u8]) -> Option<(MarkKind, Dialect)> { + let (dialect, rest) = if let Some(rest) = payload.strip_prefix(b"133;") { + (Dialect::Osc133, rest) + } else { + (Dialect::Osc633, payload.strip_prefix(b"633;")?) + }; + let kind = *rest.first()?; + // A letter must stand alone or introduce `;`-separated parameters; + // `133;Abc` is not a prompt-start with noise, it is not ours at all. + let params = match rest.get(1) { + None => &b""[..], + Some(b';') => &rest[2..], + Some(_) => return None, + }; + let mark = match kind { + b'A' => MarkKind::PromptStart, + b'B' => MarkKind::InputStart, + b'C' => MarkKind::OutputStart, + b'D' => MarkKind::Finished(parse_exit(params)), + b'E' if dialect == Dialect::Osc633 => MarkKind::CommandLine(decode_vscode_command(params)), + _ => return None, + }; + Some((mark, dialect)) +} + +/// The exit status on a `D` marker. Absent is normal: plenty of shells send +/// a bare `D`, and `D;` with nothing after it means the same thing. +fn parse_exit(params: &[u8]) -> Option { + let field = params.split(|&b| b == b';').next()?; + if field.is_empty() { + return None; + } + std::str::from_utf8(field).ok()?.trim().parse::().ok() +} + +/// `633;E` escapes its command line as `\xHH`, plus `\\` for a literal +/// backslash, so that a `;` inside the command cannot end the parameter. +fn decode_vscode_command(params: &[u8]) -> String { + let field = params.split(|&b| b == b';').next().unwrap_or(b""); + let mut out = Vec::with_capacity(field.len()); + let mut i = 0; + while i < field.len() { + if field[i] == b'\\' && i + 1 < field.len() { + match field[i + 1] { + b'x' | b'X' if i + 3 < field.len() => { + let hex = std::str::from_utf8(&field[i + 2..i + 4]).ok(); + match hex.and_then(|h| u8::from_str_radix(h, 16).ok()) { + Some(byte) => { + out.push(byte); + i += 4; + } + None => { + out.push(field[i]); + i += 1; + } + } + } + b'\\' => { + out.push(b'\\'); + i += 2; + } + _ => { + out.push(field[i]); + i += 1; + } + } + } else { + out.push(field[i]); + i += 1; + } + } + String::from_utf8_lossy(&out).into_owned() +} + +/// Retain the tail of `data` when it ends inside an unterminated OSC, so the +/// next chunk can be scanned as if the split had not happened. +/// +/// Returns the byte offset the tail starts at, or `None` when the chunk does +/// not end mid-sequence. Only an OSC introducer counts: a trailing `ESC` +/// alone is also retained, since the `]` may be the next chunk's first byte. +pub fn unterminated_osc_tail(data: &[u8]) -> Option { + // Scan backwards for the last ESC; if any terminator follows it, the + // sequence closed inside this chunk and nothing needs carrying. + let esc = data.iter().rposition(|&b| b == 0x1b)?; + if esc + 1 < data.len() && data[esc + 1] != b']' { + return None; + } + let mut i = esc + 2; + while i < data.len() { + if data[i] == 0x07 || (data[i] == 0x1b && i + 1 < data.len() && data[i + 1] == b'\\') { + return None; + } + i += 1; + } + (data.len() - esc <= CARRY_MAX).then_some(esc) +} + +/// What the journal state machine is waiting for. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +enum State { + /// No marker seen yet, or the last command has finished. + #[default] + Idle, + /// Saw `A`: a prompt is being drawn. + Prompt, + /// Saw `B`: the region from here to the next `C` is the command line. + Input { seq: u64, col: u16 }, + /// Saw `C`: a command is running and `open` is its record. + Running, +} + +/// A PTY's command history: a bounded ring of finished commands plus whatever +/// is running now. +pub struct CommandJournal { + records: VecDeque, + state: State, + /// The record for the running command, moved into `records` on `D`. + open: Option, + /// A `633;E` command line seen since the last `C`, which beats reading + /// the command back off the grid. + declared_command: Option, + /// Index the next command will take. Never reused, never rewound. + next_index: u64, + /// Lowest index still retained; rises as the ring evicts. + oldest_index: u64, + /// The dialect this PTY's shell speaks, latched on the first marker. + dialect: Option, + max_records: usize, + max_command: usize, +} + +impl Default for CommandJournal { + fn default() -> Self { + Self { + records: VecDeque::new(), + state: State::Idle, + open: None, + declared_command: None, + next_index: 0, + oldest_index: 0, + dialect: None, + max_records: env_usize("BLIT_TERM_JOURNAL_MAX", 256).max(1), + max_command: command_max(), + } + } +} + +/// Everything the journal needs to know about the terminal when a marker +/// lands: where the cursor is, and how to read the text a `B`..`C` region +/// left on the grid. +pub struct MarkContext<'a> { + pub cursor_seq: u64, + pub cursor_col: u16, + /// Text from `(seq, col)` through the end of row `end_seq`, for + /// recovering a command line off the grid. + pub read: &'a dyn Fn(u64, u16, u64) -> String, +} + +impl CommandJournal { + pub fn next_index(&self) -> u64 { + self.next_index + } + + pub fn oldest_index(&self) -> u64 { + self.oldest_index + } + + /// Every record, oldest first, with the running one last. + pub fn iter(&self) -> impl Iterator { + self.records.iter().chain(self.open.iter()) + } + + pub fn get(&self, index: u64) -> Option<&CommandRecord> { + self.iter().find(|r| r.index == index) + } + + /// The newest record, running or not. + pub fn latest(&self) -> Option<&CommandRecord> { + self.open.as_ref().or_else(|| self.records.back()) + } + + /// The running command's index, if one is running. + pub fn running_index(&self) -> Option { + self.open.as_ref().map(|r| r.index) + } + + /// A snapshot of `index` with the live fields filled in: a running + /// command's output end is wherever the terminal is now, and any record + /// whose start has fallen out of the scrollback says so. + pub fn snapshot(&self, index: u64, cursor_seq: u64, oldest_seq: u64) -> Option { + let mut record = self.get(index)?.clone(); + if record.running() { + record.end_seq = cursor_seq.max(record.start_seq); + } + if record.start_seq < oldest_seq { + record.flags |= RECORD_EVICTED; + } + Some(record) + } + + /// Apply one marker. `ctx` must describe the terminal *after* every byte + /// preceding the marker has been processed. + pub fn apply(&mut self, mark: &SemanticMark, ctx: &MarkContext<'_>) { + // A shell that speaks both dialects would otherwise open two records + // per command. First one wins; `633;E` is additive and always taken, + // since it only ever supplies text the other dialect lacks. + if !matches!(mark.kind, MarkKind::CommandLine(_)) { + match self.dialect { + None => self.dialect = Some(mark.dialect), + Some(latched) if latched != mark.dialect => return, + Some(_) => {} + } + } + match &mark.kind { + MarkKind::PromptStart => { + // A prompt while a command is running means the command ended + // without saying so — Ctrl-C, a shell that only emits A, or a + // reset. Close it honestly rather than leaving it running + // forever. + self.close_open(ctx.cursor_seq, None, RECORD_INCOMPLETE); + self.declared_command = None; + self.state = State::Prompt; + } + MarkKind::InputStart => { + self.state = State::Input { + seq: ctx.cursor_seq, + col: ctx.cursor_col, + }; + } + MarkKind::OutputStart => { + // Two `C`s without a `D` between them: the first command's + // output ends where the second's begins. + self.close_open(ctx.cursor_seq, None, RECORD_INCOMPLETE); + let command = self.recover_command(ctx); + let mut flags = RECORD_RUNNING; + if command.is_empty() { + flags |= RECORD_NO_COMMAND; + } + let index = self.next_index; + self.next_index += 1; + self.open = Some(CommandRecord { + index, + flags, + exit_code: 0, + start_seq: ctx.cursor_seq, + end_seq: ctx.cursor_seq, + started_ms: now_ms(), + ended_ms: 0, + command, + }); + self.declared_command = None; + self.state = State::Running; + } + MarkKind::Finished(exit) => { + // A `D` with nothing open is a shell reporting the status of + // something it never announced starting. There is no record + // to attach it to, so it is dropped. + self.close_open(ctx.cursor_seq, *exit, 0); + self.state = State::Idle; + } + MarkKind::CommandLine(cmd) => { + let mut cmd = cmd.clone(); + truncate_utf8(&mut cmd, self.max_command); + self.declared_command = Some(cmd); + } + } + } + + /// The command line for the record being opened: what the shell declared, + /// else whatever sits between the `B` marker and here. + fn recover_command(&mut self, ctx: &MarkContext<'_>) -> String { + if let Some(cmd) = self.declared_command.take() { + return cmd; + } + let State::Input { seq, col } = self.state else { + // `C` with no preceding `B`: there is no input region to read. + return String::new(); + }; + // The command line ends where the cursor is now, less the newline + // that ran it — which has already moved the cursor to the next row, + // so reading to the row above is reading exactly the typed text. + let end_seq = if ctx.cursor_col == 0 && ctx.cursor_seq > seq { + ctx.cursor_seq - 1 + } else { + ctx.cursor_seq + }; + let mut text = (ctx.read)(seq, col, end_seq); + // A multi-line command (a continuation, a here-doc) arrives with the + // rows it occupied; keep them, they are the command. + let trimmed = text.trim_end().len(); + text.truncate(trimmed); + truncate_utf8(&mut text, self.max_command); + text + } + + /// Finish the running command, if any, and retire it into the ring. + fn close_open(&mut self, end_seq: u64, exit: Option, extra_flags: u8) { + let Some(mut record) = self.open.take() else { + return; + }; + record.flags &= !RECORD_RUNNING; + record.flags |= extra_flags; + record.end_seq = end_seq.max(record.start_seq); + record.ended_ms = now_ms(); + if let Some(code) = exit { + record.flags |= RECORD_HAS_EXIT; + record.exit_code = code; + } + self.records.push_back(record); + while self.records.len() > self.max_records { + self.records.pop_front(); + self.oldest_index += 1; + } + } + + /// The PTY's process is gone. A command still running when that happens + /// never gets its `D`, so close it and say why. + pub fn note_pty_exit(&mut self, end_seq: u64) { + self.close_open(end_seq, None, RECORD_INCOMPLETE | RECORD_PTY_EXITED); + self.state = State::Idle; + } + + /// A restarted PTY is a new shell in the same slot; its predecessor's + /// commands describe output that has been reset away. + pub fn reset(&mut self) { + self.records.clear(); + self.open = None; + self.declared_command = None; + self.state = State::Idle; + self.dialect = None; + self.oldest_index = self.next_index; + } +} + +/// A client blocked on a command finishing (`C2S_TERM_JOURNAL_WAIT`). +/// +/// Waiting is server-side on purpose: a client polling a journal to find out +/// whether `make` is done is the pattern this whole family exists to remove. +pub struct Waiter { + pub client_id: u64, + pub nonce: u16, + pub pty_id: u16, + /// The command being waited on. `None` until a command is running to + /// attach to — `JOURNAL_INDEX_LATEST` on an idle shell means "the next + /// one", so the index is latched the moment one starts and never moves + /// again. + pub index: Option, + pub deadline: std::time::Instant, +} + +/// Per connection, so a client cannot park unbounded state on the server. +pub const MAX_WAITERS_PER_CLIENT: usize = 32; + +/// Longest a `C2S_TERM_JOURNAL_WAIT` may block before answering with the +/// record as it stands. +pub const WAIT_TIMEOUT_MAX_MS: u32 = 24 * 60 * 60 * 1000; + +/// What should happen to a waiter now. +pub enum WaitOutcome { + /// Nothing yet. + Pending, + /// Answer with this record and drop the waiter. + Ready(CommandRecord), + /// Answer `NOT_FOUND` and drop the waiter: the PTY is gone, or the record + /// was evicted before the wait could see it finish. + Gone, +} + +impl Waiter { + /// Decide a waiter against a PTY's journal. + /// + /// `journal` is `None` when the PTY itself has gone away. + pub fn poll( + &mut self, + journal: Option<&CommandJournal>, + cursor_seq: u64, + oldest_seq: u64, + now: std::time::Instant, + ) -> WaitOutcome { + let Some(journal) = journal else { + return WaitOutcome::Gone; + }; + let expired = now >= self.deadline; + if self.index.is_none() { + self.index = journal.running_index(); + } + let Some(index) = self.index else { + // Still nothing running. On timeout there is no record to hand + // back, so say so rather than invent an empty one. + return if expired { + WaitOutcome::Gone + } else { + WaitOutcome::Pending + }; + }; + match journal.snapshot(index, cursor_seq, oldest_seq) { + Some(record) if !record.running() => WaitOutcome::Ready(record), + // Running: answer on timeout with the record as it stands, still + // flagged RUNNING so the caller can tell it timed out. + Some(record) if expired => WaitOutcome::Ready(record), + Some(_) => WaitOutcome::Pending, + // Not in the ring. Below the floor it was evicted; above it, the + // command has not started yet and is still worth waiting for. + None if index < journal.oldest_index() || expired => WaitOutcome::Gone, + None => WaitOutcome::Pending, + } + } +} + +/// Truncate to at most `max` bytes without splitting a character. +fn truncate_utf8(s: &mut String, max: usize) { + if s.len() <= max { + return; + } + let mut end = max; + while end > 0 && !s.is_char_boundary(end) { + end -= 1; + } + s.truncate(end); +} + +#[cfg(test)] +mod tests { + use super::*; + + fn ctx(seq: u64, col: u16) -> MarkContext<'static> { + MarkContext { + cursor_seq: seq, + cursor_col: col, + read: &|_, _, _| String::new(), + } + } + + fn mark(kind: MarkKind) -> SemanticMark { + SemanticMark { + kind, + dialect: Dialect::Osc133, + at: 0, + } + } + + fn mark633(kind: MarkKind) -> SemanticMark { + SemanticMark { + kind, + dialect: Dialect::Osc633, + at: 0, + } + } + + // ── Payload parsing ──────────────────────────────────────────────── + + #[test] + fn parses_the_four_standard_markers() { + assert_eq!( + parse_mark(b"133;A"), + Some((MarkKind::PromptStart, Dialect::Osc133)) + ); + assert_eq!( + parse_mark(b"133;B"), + Some((MarkKind::InputStart, Dialect::Osc133)) + ); + assert_eq!( + parse_mark(b"133;C"), + Some((MarkKind::OutputStart, Dialect::Osc133)) + ); + assert_eq!( + parse_mark(b"133;D;0"), + Some((MarkKind::Finished(Some(0)), Dialect::Osc133)) + ); + assert_eq!( + parse_mark(b"133;D;130"), + Some((MarkKind::Finished(Some(130)), Dialect::Osc133)) + ); + } + + #[test] + fn tolerates_a_finish_marker_with_no_status() { + // Both spellings are common in the wild. + assert_eq!( + parse_mark(b"133;D"), + Some((MarkKind::Finished(None), Dialect::Osc133)) + ); + assert_eq!( + parse_mark(b"133;D;"), + Some((MarkKind::Finished(None), Dialect::Osc133)) + ); + } + + #[test] + fn tolerates_a_malformed_status() { + for payload in [ + &b"133;D;banana"[..], + b"133;D;99999999999999999999", + b"133;D;1.5", + b"133;D; ;aid=3", + ] { + assert_eq!( + parse_mark(payload), + Some((MarkKind::Finished(None), Dialect::Osc133)), + "{:?} should degrade to an unknown status, not be dropped", + String::from_utf8_lossy(payload) + ); + } + } + + #[test] + fn ignores_trailing_marker_parameters() { + assert_eq!( + parse_mark(b"133;A;aid=17;cl=m"), + Some((MarkKind::PromptStart, Dialect::Osc133)) + ); + assert_eq!( + parse_mark(b"133;D;3;aid=17"), + Some((MarkKind::Finished(Some(3)), Dialect::Osc133)) + ); + } + + #[test] + fn ignores_markers_that_are_not_ours() { + for payload in [ + &b"7;file:///tmp"[..], + b"0;a title", + b"133", + b"133;", + b"133;P;k=i", + b"133;L", + b"133;Z", + b"133;Abc", + b"633;P;Cwd=/tmp", + b"1337;File=inline", + b"", + ] { + assert_eq!( + parse_mark(payload), + None, + "{:?} is not a semantic-prompt marker", + String::from_utf8_lossy(payload) + ); + } + } + + #[test] + fn parses_the_vscode_command_line() { + assert_eq!( + parse_mark(b"633;E;ls -la"), + Some((MarkKind::CommandLine("ls -la".into()), Dialect::Osc633)) + ); + // `;` and the escape character itself are hex-escaped by the shell + // integration script so they cannot end the parameter. + assert_eq!( + parse_mark(b"633;E;echo a\\x3bb"), + Some((MarkKind::CommandLine("echo a;b".into()), Dialect::Osc633)) + ); + assert_eq!( + parse_mark(b"633;E;a\\\\b"), + Some((MarkKind::CommandLine("a\\b".into()), Dialect::Osc633)) + ); + // A truncated escape is data, not a panic. + assert_eq!( + parse_mark(b"633;E;tail\\x"), + Some((MarkKind::CommandLine("tail\\x".into()), Dialect::Osc633)) + ); + // `E` is VS Code's alone; 133 has no such marker. + assert_eq!(parse_mark(b"133;E;ls"), None); + } + + // ── Split sequences ──────────────────────────────────────────────── + + #[test] + fn carries_a_sequence_split_across_chunks() { + assert_eq!(unterminated_osc_tail(b"output\x1b]133;C"), Some(6)); + assert_eq!(unterminated_osc_tail(b"output\x1b]"), Some(6)); + assert_eq!(unterminated_osc_tail(b"output\x1b"), Some(6)); + } + + #[test] + fn carries_nothing_when_the_sequence_completed() { + assert_eq!(unterminated_osc_tail(b"\x1b]133;C\x07more"), None); + assert_eq!(unterminated_osc_tail(b"\x1b]133;C\x1b\\"), None); + assert_eq!(unterminated_osc_tail(b"plain output"), None); + // A CSI is somebody else's problem; only OSC needs the carry. + assert_eq!(unterminated_osc_tail(b"\x1b[0m"), None); + } + + #[test] + fn refuses_to_carry_an_unbounded_tail() { + let mut data = vec![0x1b, b']']; + data.extend(std::iter::repeat_n(b'x', CARRY_MAX)); + assert_eq!( + unterminated_osc_tail(&data), + None, + "a sequence that never terminates must not grow the carry buffer" + ); + } + + // ── State machine ────────────────────────────────────────────────── + + fn run(marks: &[(SemanticMark, u64)]) -> CommandJournal { + let mut journal = CommandJournal::default(); + for (mark, seq) in marks { + journal.apply(mark, &ctx(*seq, 0)); + } + journal + } + + #[test] + fn a_full_cycle_produces_one_finished_record() { + let journal = run(&[ + (mark(MarkKind::PromptStart), 10), + (mark(MarkKind::InputStart), 10), + (mark(MarkKind::OutputStart), 11), + (mark(MarkKind::Finished(Some(0))), 20), + ]); + let records: Vec<_> = journal.iter().collect(); + assert_eq!(records.len(), 1); + assert_eq!(records[0].index, 0); + assert_eq!(records[0].start_seq, 11); + assert_eq!(records[0].end_seq, 20); + assert_eq!(records[0].exit(), Some(0)); + assert!(!records[0].running()); + assert_eq!(journal.next_index(), 1); + } + + #[test] + fn a_running_command_is_visible_before_it_finishes() { + let journal = run(&[ + (mark(MarkKind::PromptStart), 5), + (mark(MarkKind::InputStart), 5), + (mark(MarkKind::OutputStart), 6), + ]); + let latest = journal.latest().expect("a record"); + assert!(latest.running()); + assert_eq!(journal.running_index(), Some(0)); + // Its end follows the terminal until it finishes. + let snap = journal.snapshot(0, 42, 0).expect("snapshot"); + assert_eq!(snap.end_seq, 42); + } + + #[test] + fn output_start_without_input_start_still_records() { + let journal = run(&[ + (mark(MarkKind::PromptStart), 1), + (mark(MarkKind::OutputStart), 2), + (mark(MarkKind::Finished(Some(1))), 5), + ]); + let record = journal.get(0).expect("a record"); + assert_eq!(record.flags & RECORD_NO_COMMAND, RECORD_NO_COMMAND); + assert_eq!(record.command, ""); + assert_eq!(record.exit(), Some(1)); + } + + #[test] + fn a_finish_with_nothing_running_is_dropped() { + let journal = run(&[ + (mark(MarkKind::Finished(Some(0))), 3), + (mark(MarkKind::Finished(Some(0))), 4), + ]); + assert_eq!(journal.iter().count(), 0); + assert_eq!(journal.next_index(), 0); + } + + #[test] + fn a_second_finish_does_not_reopen_the_record() { + let journal = run(&[ + (mark(MarkKind::OutputStart), 1), + (mark(MarkKind::Finished(Some(0))), 4), + (mark(MarkKind::Finished(Some(7))), 9), + ]); + assert_eq!(journal.iter().count(), 1); + assert_eq!(journal.get(0).unwrap().exit(), Some(0)); + assert_eq!(journal.get(0).unwrap().end_seq, 4); + } + + #[test] + fn a_new_prompt_closes_an_interrupted_command() { + // Ctrl-C: the shell redraws its prompt and never sends `D`. + let journal = run(&[ + (mark(MarkKind::OutputStart), 3), + (mark(MarkKind::PromptStart), 8), + ]); + let record = journal.get(0).expect("a record"); + assert!( + !record.running(), + "an interrupted command is not still running" + ); + assert_eq!(record.flags & RECORD_INCOMPLETE, RECORD_INCOMPLETE); + assert_eq!(record.exit(), None); + assert_eq!(record.end_seq, 8); + } + + #[test] + fn back_to_back_output_markers_split_into_two_records() { + let journal = run(&[ + (mark(MarkKind::OutputStart), 2), + (mark(MarkKind::OutputStart), 6), + (mark(MarkKind::Finished(Some(0))), 9), + ]); + let records: Vec<_> = journal.iter().collect(); + assert_eq!(records.len(), 2); + assert_eq!((records[0].start_seq, records[0].end_seq), (2, 6)); + assert_eq!(records[0].flags & RECORD_INCOMPLETE, RECORD_INCOMPLETE); + assert_eq!((records[1].start_seq, records[1].end_seq), (6, 9)); + assert_eq!(records[1].exit(), Some(0)); + } + + #[test] + fn markers_arriving_out_of_order_never_wedge_the_machine() { + // Every permutation of a cycle's markers, replayed twice: whatever + // the shell does, the journal must still accept a clean cycle after. + let kinds = [ + MarkKind::Finished(Some(0)), + MarkKind::OutputStart, + MarkKind::InputStart, + MarkKind::PromptStart, + ]; + for a in 0..4 { + for b in 0..4 { + for c in 0..4 { + let mut journal = CommandJournal::default(); + for (i, k) in [a, b, c].iter().enumerate() { + journal.apply(&mark(kinds[*k].clone()), &ctx(i as u64, 0)); + } + let before = journal.next_index(); + for (i, k) in [ + MarkKind::PromptStart, + MarkKind::InputStart, + MarkKind::OutputStart, + MarkKind::Finished(Some(42)), + ] + .into_iter() + .enumerate() + { + journal.apply(&mark(k), &ctx(100 + i as u64, 0)); + } + let last = journal.latest().expect("the clean cycle recorded"); + assert_eq!( + last.exit(), + Some(42), + "a clean cycle after [{a},{b},{c}] must still record" + ); + assert!(!last.running()); + assert!(journal.next_index() > before); + } + } + } + } + + #[test] + fn indices_are_stable_while_the_ring_evicts() { + unsafe { std::env::set_var("BLIT_TERM_JOURNAL_MAX", "3") }; + let mut journal = CommandJournal::default(); + unsafe { std::env::remove_var("BLIT_TERM_JOURNAL_MAX") }; + for i in 0..10u64 { + journal.apply(&mark(MarkKind::OutputStart), &ctx(i * 2, 0)); + journal.apply(&mark(MarkKind::Finished(Some(0))), &ctx(i * 2 + 1, 0)); + } + assert_eq!(journal.next_index(), 10); + assert_eq!(journal.oldest_index(), 7); + assert_eq!(journal.iter().count(), 3); + let indices: Vec = journal.iter().map(|r| r.index).collect(); + assert_eq!(indices, vec![7, 8, 9], "indices are never renumbered"); + assert!(journal.get(3).is_none(), "evicted records are gone"); + assert_eq!(journal.get(9).map(|r| r.index), Some(9)); + } + + #[test] + fn evicted_output_is_flagged_at_read_time() { + let journal = run(&[ + (mark(MarkKind::OutputStart), 10), + (mark(MarkKind::Finished(Some(0))), 20), + ]); + let fresh = journal.snapshot(0, 30, 5).expect("snapshot"); + assert_eq!(fresh.flags & RECORD_EVICTED, 0); + let stale = journal.snapshot(0, 3000, 500).expect("snapshot"); + assert_eq!(stale.flags & RECORD_EVICTED, RECORD_EVICTED); + } + + #[test] + fn a_pty_exit_closes_whatever_was_running() { + let mut journal = run(&[(mark(MarkKind::OutputStart), 4)]); + journal.note_pty_exit(9); + let record = journal.get(0).expect("a record"); + assert!(!record.running()); + assert_eq!(record.flags & RECORD_PTY_EXITED, RECORD_PTY_EXITED); + assert_eq!(record.end_seq, 9); + } + + #[test] + fn a_restart_retires_the_old_shells_commands() { + let mut journal = run(&[ + (mark(MarkKind::OutputStart), 1), + (mark(MarkKind::Finished(Some(0))), 2), + ]); + journal.reset(); + assert_eq!(journal.iter().count(), 0); + assert_eq!( + journal.oldest_index(), + journal.next_index(), + "indices keep going up so a stale client index cannot alias" + ); + journal.apply(&mark(MarkKind::OutputStart), &ctx(0, 0)); + assert_eq!(journal.latest().unwrap().index, 1); + } + + #[test] + fn a_shell_speaking_both_dialects_is_not_counted_twice() { + let mut journal = CommandJournal::default(); + for (kind, dialect633) in [ + (MarkKind::PromptStart, true), + (MarkKind::InputStart, true), + (MarkKind::OutputStart, true), + (MarkKind::Finished(Some(0)), true), + ] { + journal.apply(&mark(kind.clone()), &ctx(1, 0)); + if dialect633 { + journal.apply(&mark633(kind), &ctx(1, 0)); + } + } + assert_eq!( + journal.iter().count(), + 1, + "the second dialect must not open a second record" + ); + } + + #[test] + fn a_declared_command_line_beats_reading_the_grid() { + let mut journal = CommandJournal::default(); + journal.apply(&mark(MarkKind::PromptStart), &ctx(0, 0)); + journal.apply(&mark(MarkKind::InputStart), &ctx(0, 2)); + journal.apply( + &mark633(MarkKind::CommandLine("make -j8".into())), + &ctx(0, 2), + ); + journal.apply( + &SemanticMark { + kind: MarkKind::OutputStart, + dialect: Dialect::Osc133, + at: 0, + }, + &MarkContext { + cursor_seq: 1, + cursor_col: 0, + read: &|_, _, _| "$ make -j8".into(), + }, + ); + assert_eq!(journal.latest().unwrap().command, "make -j8"); + } + + #[test] + fn a_command_line_is_read_back_off_the_grid() { + let mut journal = CommandJournal::default(); + journal.apply(&mark(MarkKind::PromptStart), &ctx(4, 0)); + journal.apply(&mark(MarkKind::InputStart), &ctx(4, 2)); + journal.apply( + &mark(MarkKind::OutputStart), + &MarkContext { + cursor_seq: 5, + cursor_col: 0, + read: &|s, c, e| { + assert_eq!( + (s, c, e), + (4, 2, 4), + "read the typed region, not the prompt" + ); + "cargo test ".into() + }, + }, + ); + assert_eq!(journal.latest().unwrap().command, "cargo test"); + } + + #[test] + fn an_enormous_command_line_is_bounded() { + let mut journal = CommandJournal { + max_command: 8, + ..CommandJournal::default() + }; + journal.apply(&mark633(MarkKind::CommandLine("é".repeat(50))), &ctx(0, 0)); + journal.apply(&mark(MarkKind::OutputStart), &ctx(0, 0)); + let command = &journal.latest().unwrap().command; + assert!(command.len() <= 8); + assert!(command.chars().all(|c| c == 'é'), "no split character"); + } + + // ── Waiting ──────────────────────────────────────────────────────── + + fn waiter(index: Option, timeout_ms: u64) -> Waiter { + Waiter { + client_id: 1, + nonce: 7, + pty_id: 2, + index, + deadline: std::time::Instant::now() + std::time::Duration::from_millis(timeout_ms), + } + } + + fn expired_waiter(index: Option) -> Waiter { + let mut w = waiter(index, 0); + w.deadline = std::time::Instant::now() - std::time::Duration::from_secs(1); + w + } + + #[test] + fn a_waiter_resolves_when_the_command_finishes() { + let mut journal = run(&[(mark(MarkKind::OutputStart), 3)]); + let mut w = waiter(None, 60_000); + let now = std::time::Instant::now(); + assert!(matches!( + w.poll(Some(&journal), 5, 0, now), + WaitOutcome::Pending + )); + assert_eq!(w.index, Some(0), "the running command is latched"); + + journal.apply(&mark(MarkKind::Finished(Some(3))), &ctx(9, 0)); + match w.poll(Some(&journal), 9, 0, now) { + WaitOutcome::Ready(record) => assert_eq!(record.exit(), Some(3)), + _ => panic!("a finished command must resolve its waiter"), + } + } + + #[test] + fn a_waiter_on_an_idle_shell_latches_the_next_command() { + let mut journal = CommandJournal::default(); + let mut w = waiter(None, 60_000); + let now = std::time::Instant::now(); + assert!(matches!( + w.poll(Some(&journal), 0, 0, now), + WaitOutcome::Pending + )); + assert_eq!(w.index, None); + + journal.apply(&mark(MarkKind::OutputStart), &ctx(1, 0)); + assert!(matches!( + w.poll(Some(&journal), 1, 0, now), + WaitOutcome::Pending + )); + assert_eq!(w.index, Some(0)); + } + + #[test] + fn a_timeout_hands_back_the_record_still_running() { + let journal = run(&[(mark(MarkKind::OutputStart), 3)]); + let mut w = expired_waiter(None); + match w.poll(Some(&journal), 40, 0, std::time::Instant::now()) { + WaitOutcome::Ready(record) => { + assert!(record.running(), "a timeout does not finish the command"); + assert_eq!(record.end_seq, 40, "its output still runs to the bottom"); + } + _ => panic!("a timeout must answer, not hang"), + } + } + + #[test] + fn a_waiter_gives_up_on_a_vanished_pty_or_record() { + let journal = CommandJournal::default(); + assert!(matches!( + waiter(None, 60_000).poll(None, 0, 0, std::time::Instant::now()), + WaitOutcome::Gone + )); + assert!(matches!( + expired_waiter(None).poll(Some(&journal), 0, 0, std::time::Instant::now()), + WaitOutcome::Gone + )); + } + + #[test] + fn a_waiter_on_an_evicted_record_gives_up_rather_than_hanging() { + unsafe { std::env::set_var("BLIT_TERM_JOURNAL_MAX", "1") }; + let mut journal = CommandJournal::default(); + unsafe { std::env::remove_var("BLIT_TERM_JOURNAL_MAX") }; + for i in 0..4u64 { + journal.apply(&mark(MarkKind::OutputStart), &ctx(i, 0)); + journal.apply(&mark(MarkKind::Finished(Some(0))), &ctx(i, 0)); + } + let mut w = waiter(Some(0), 60_000); + assert!(matches!( + w.poll(Some(&journal), 9, 0, std::time::Instant::now()), + WaitOutcome::Gone + )); + } + + #[test] + fn a_waiter_on_a_future_index_keeps_waiting() { + let journal = run(&[ + (mark(MarkKind::OutputStart), 1), + (mark(MarkKind::Finished(Some(0))), 2), + ]); + let mut w = waiter(Some(5), 60_000); + assert!(matches!( + w.poll(Some(&journal), 3, 0, std::time::Instant::now()), + WaitOutcome::Pending + )); + } + + #[test] + fn a_terminal_with_no_shell_integration_stays_empty() { + let journal = CommandJournal::default(); + assert_eq!(journal.next_index(), 0); + assert_eq!(journal.iter().count(), 0); + assert_eq!(journal.latest(), None); + assert_eq!(journal.snapshot(0, 0, 0), None); + } +} diff --git a/crates/server/src/lib.rs b/crates/server/src/lib.rs index b3e901b2..2e8f9083 100644 --- a/crates/server/src/lib.rs +++ b/crates/server/src/lib.rs @@ -1,4 +1,6 @@ -use blit_alacritty::{SearchResult as AlacrittySearchResult, TerminalDriver as AlacrittyDriver}; +use blit_alacritty::{ + SearchResult as AlacrittySearchResult, SeqText, TerminalDriver as AlacrittyDriver, +}; use blit_compositor::{ CompositorCommand, CompositorEvent, CompositorHandle, TouchPhase, TouchPoint, }; @@ -32,24 +34,25 @@ use blit_remote::{ C2S_SURFACE_POINTER, C2S_SURFACE_POINTER_AXIS, C2S_SURFACE_POINTER_AXIS2, C2S_SURFACE_PREEDIT, C2S_SURFACE_RESIZE, C2S_SURFACE_SUBSCRIBE, C2S_SURFACE_TEXT, C2S_SURFACE_TOUCH, C2S_SURFACE_UNSUBSCRIBE, C2S_TERM_CWD, C2S_UNSUBSCRIBE, CAPTURE_FORMAT_AVIF, - CAPTURE_FORMAT_PNG, CLIENT_FEATURE_SURFACE_TIMESTAMP_SUB_US, CREATE2_HAS_COMMAND, - CREATE2_HAS_CWD, CREATE2_HAS_DEADLINE, CREATE2_HAS_SRC_PTY, CREATE2_WANT_STATUS, - FEATURE_CLIENT_CONTROL, FEATURE_COPY_RANGE, FEATURE_CREATE_NONCE, FEATURE_CREATE_STATUS, - FEATURE_KILL_MODE, FEATURE_PTY_DEADLINE, FEATURE_RESIZE_BATCH, FEATURE_RESTART, - FEATURE_SCROLL_BY, FrameState, KICK_REASON_MAX, KILL_LEADER_ONLY, READ_ANSI, READ_TAIL, - REMOTE_INPUT_POINTER, REMOTE_INPUT_TOUCH, S2C_CLOSED, S2C_CREATED, S2C_CREATED_N, S2C_LIST, - S2C_PING, S2C_QUIT, S2C_READY, S2C_SEARCH_RESULTS, S2C_SURFACE_CAPTURE, S2C_SURFACE_LIST, - S2C_TEXT, S2C_TITLE, STATUS_BUDGET, STATUS_INVALID, STATUS_NOT_FOUND, STATUS_OK, STATUS_OTHER, - STATUS_TOO_LARGE, SURFACE_FRAME_CODEC_H264, SURFACE_FRAME_FLAG_KEYFRAME, - SURFACE_POINTER_AXIS2_LEN, SURFACE_POINTER_DOWN, SURFACE_POINTER_LEAVE, SURFACE_POINTER_MOVE, - SURFACE_POINTER_UP, SURFACE_TOUCH_CANCEL, SURFACE_TOUCH_DISABLE, SURFACE_TOUCH_DOWN, - SURFACE_TOUCH_ENABLE, SURFACE_TOUCH_MOTION, SURFACE_TOUCH_UP, build_update_msg, - clamp_cursor_rect, msg_hello, msg_kick_result, msg_kicked, msg_s2c_client_list, - msg_s2c_clipboard_content, msg_s2c_clipboard_list, msg_s2c_clipboard_owner, - msg_s2c_scroll_offset, msg_s2c_surface_remote_input, msg_s2c_used_rows, msg_surface_activated, - msg_surface_app_id, msg_surface_created, msg_surface_destroyed, msg_surface_encoder, - msg_surface_frame, msg_surface_frame_precise, msg_surface_resized, msg_surface_text_input, - msg_surface_title, msg_term_cwd_reply, parse_surface_drag_drop, parse_surface_drag_enter, + CAPTURE_FORMAT_PNG, CLIENT_FEATURE_SURFACE_TIMESTAMP_SUB_US, CLIENT_LIST_WANT_ORIGIN, + CREATE2_HAS_COMMAND, CREATE2_HAS_CWD, CREATE2_HAS_DEADLINE, CREATE2_HAS_SRC_PTY, + CREATE2_WANT_STATUS, FEATURE_CLIENT_CONTROL, FEATURE_CLIENT_ORIGIN, FEATURE_COPY_RANGE, + FEATURE_CREATE_NONCE, FEATURE_CREATE_STATUS, FEATURE_KILL_MODE, FEATURE_PTY_DEADLINE, + FEATURE_RESIZE_BATCH, FEATURE_RESTART, FEATURE_SCROLL_BY, FrameState, KICK_REASON_MAX, + KILL_LEADER_ONLY, READ_ANSI, READ_TAIL, REMOTE_INPUT_POINTER, REMOTE_INPUT_TOUCH, S2C_CLOSED, + S2C_CREATED, S2C_CREATED_N, S2C_LIST, S2C_PING, S2C_QUIT, S2C_READY, S2C_SEARCH_RESULTS, + S2C_SURFACE_CAPTURE, S2C_SURFACE_LIST, S2C_TEXT, S2C_TITLE, STATUS_BUDGET, STATUS_INVALID, + STATUS_NOT_FOUND, STATUS_OK, STATUS_OTHER, STATUS_TOO_LARGE, SURFACE_FRAME_CODEC_H264, + SURFACE_FRAME_FLAG_KEYFRAME, SURFACE_POINTER_AXIS2_LEN, SURFACE_POINTER_DOWN, + SURFACE_POINTER_LEAVE, SURFACE_POINTER_MOVE, SURFACE_POINTER_UP, SURFACE_TOUCH_CANCEL, + SURFACE_TOUCH_DISABLE, SURFACE_TOUCH_DOWN, SURFACE_TOUCH_ENABLE, SURFACE_TOUCH_MOTION, + SURFACE_TOUCH_UP, build_update_msg, clamp_cursor_rect, msg_hello, msg_kick_result, msg_kicked, + msg_s2c_client_list, msg_s2c_client_list2, msg_s2c_clipboard_content, msg_s2c_clipboard_list, + msg_s2c_clipboard_owner, msg_s2c_scroll_offset, msg_s2c_surface_remote_input, + msg_s2c_used_rows, msg_surface_activated, msg_surface_app_id, msg_surface_created, + msg_surface_destroyed, msg_surface_encoder, msg_surface_frame, msg_surface_frame_precise, + msg_surface_origin, msg_surface_resized, msg_surface_text_input, msg_surface_title, + msg_term_cwd_reply, parse_surface_drag_drop, parse_surface_drag_enter, parse_surface_pointer_axis2, parse_surface_touch, }; #[cfg(target_os = "linux")] @@ -82,6 +85,7 @@ mod extension_jobs; pub mod extension_store; mod gpu_libs; mod ipc; +mod journal; mod kv; #[cfg(target_os = "linux")] mod media_input; @@ -480,6 +484,18 @@ trait PtyDriver: Send { ) -> String; fn total_lines(&self) -> u32; fn scrolled_lines(&self) -> u64; + /// Absolute sequence and column of the cursor + /// (docs/design/term-journal.md § Sequences). + fn cursor_seq(&self) -> (u64, u16); + /// Oldest sequence the scrollback still holds. + fn oldest_seq(&self) -> u64; + fn seq_text( + &self, + from_seq: u64, + from_col: u16, + end_seq: Option, + max_bytes: usize, + ) -> SeqText; } struct PtySearchResult { @@ -588,6 +604,24 @@ impl PtyDriver for AlacrittyDriver { fn scrolled_lines(&self) -> u64 { AlacrittyDriver::scrolled_lines(self) } + + fn cursor_seq(&self) -> (u64, u16) { + AlacrittyDriver::cursor_seq(self) + } + + fn oldest_seq(&self) -> u64 { + AlacrittyDriver::oldest_seq(self) + } + + fn seq_text( + &self, + from_seq: u64, + from_col: u16, + end_seq: Option, + max_bytes: usize, + ) -> SeqText { + AlacrittyDriver::seq_text(self, from_seq, from_col, end_seq, max_bytes) + } } // Soft backpressure thresholds. The outbox channel is unbounded so messages @@ -1307,6 +1341,9 @@ enum ConnectionOrigin { definition_revision: u64, attempt: u64, task_id: u32, + /// The durable name of a persistent definition, the label a transient + /// `ext run` carried, or empty when it had neither. + name: String, }, } @@ -1318,6 +1355,26 @@ impl ConnectionOrigin { } } + /// The same fact in the shape `S2C_CLIENT_LIST2` carries. + fn wire_origin(&self) -> blit_remote::ClientOrigin { + match self { + Self::Network => blit_remote::ClientOrigin::Network, + Self::Extension { + extension_id, + definition_revision, + attempt, + task_id, + name, + } => blit_remote::ClientOrigin::Extension { + extension_id: *extension_id, + definition_revision: *definition_revision, + attempt: *attempt, + task_id: *task_id, + name: name.clone(), + }, + } + } + fn channel_peer_name(&self, endpoint: u64) -> String { match self { Self::Network => format!("client:{endpoint:016x}"), @@ -1423,6 +1480,7 @@ impl ConnectionOptions { definition_revision: init.definition_revision, attempt: init.attempt, task_id: init.task_id, + name: init.name.to_owned(), }, profile: ConnectionProfile::IN_PROCESS, cancellation, @@ -1991,6 +2049,15 @@ struct Pty { /// tracking"). Last write wins; None until shell integration first /// reports (then `C2S_TERM_CWD` falls back to the kernel's view). osc7_cwd: Option, + /// Commands this PTY's shell announced through OSC 133 + /// (docs/design/term-journal.md). Empty and free for every shell + /// without integration. + journal: journal::CommandJournal, + /// An OSC left unterminated at the end of the last chunk, held so the + /// scan of the next one sees the whole sequence. A PTY read boundary + /// falls wherever the kernel put it, and a marker split across one would + /// otherwise be lost — silently mis-attributing a command's output. + osc_carry: Vec, } impl Pty { @@ -2035,9 +2102,20 @@ fn arm_pty_output_coalesce( *snapshot_not_before = Some((now + PTY_OUTPUT_QUIET).min(hard_deadline)); } +/// A surface's stamped application identity, mirroring the compositor's +/// `AppIdentity`. Absent for anything on the shared Wayland socket. +#[derive(Debug, Clone)] +struct SurfaceOrigin { + sandbox_engine: String, + app_id: String, + instance_id: String, +} + struct CachedSurfaceInfo { surface_id: u16, parent_id: u16, + /// Stamped identity, as opposed to the self-asserted `app_id` below. + origin: Option, width: u16, height: u16, /// The composited size in surface-logical pixels, as last reported by @@ -3544,6 +3622,15 @@ impl From>> for TrackedAudioSender { } } +/// One live `C2S_CLIENT_WATCH`. +struct ClientCatalogWatch { + /// Set when the watch was opened with `CLIENT_LIST_WANT_ORIGIN`. A watch + /// keeps the shape it was opened with for its whole life, so a client that + /// asked for origins never has to re-detect which reply it is reading. + want_origin: bool, + last: Vec, +} + struct ClientState { tx: TrackedOutboxSender, /// Native-channel frames retain their pair admission reservation while @@ -3581,11 +3668,15 @@ struct ClientState { inbound_bytes_seen: u64, inbound_sampled_at: Instant, inbound_bytes_per_sec: u64, - /// Live catalog nonce → last encoded snapshot sent under that nonce. - /// Comparing the deterministic encoding avoids pushing unchanged lists on - /// every delivery tick. - client_catalog_watches: FxHashMap>, + /// Live catalog nonce → the shape that watch was opened with and the last + /// encoded snapshot sent under it. Comparing the deterministic encoding + /// avoids pushing unchanged lists on every delivery tick. + client_catalog_watches: FxHashMap, connected_at: Instant, + /// What opened this connection, kept so the catalog can say so. Captured + /// at accept time: an attempt is named by the definition it started from, + /// not by whatever that definition says a minute later. + origin: ConnectionOrigin, /// Connection-scoped live resources outside terminal/surface state. /// Rebuilt after each family message from the authoritative registries. aux_subscriptions: Vec, @@ -6002,6 +6093,10 @@ struct Session { /// Direct touch is an implicit-grab sequence. Only this connection may /// extend it until all of its contacts are up or it cancels. surface_touch_owner: Option, + /// Clients blocked in `C2S_TERM_JOURNAL_WAIT`. Session-scoped because + /// what they are waiting on is a PTY, which outlives any one connection + /// (docs/design/term-journal.md § Waiting). + journal_waiters: Vec, #[cfg(target_os = "linux")] recent_surface_focus: HashMap, #[cfg(target_os = "linux")] @@ -6177,6 +6272,7 @@ impl Session { surface_frames_sent: 0, surface_inputs: HashMap::new(), surface_touch_owner: None, + journal_waiters: Vec::new(), #[cfg(target_os = "linux")] recent_surface_focus: HashMap::new(), #[cfg(target_os = "linux")] @@ -6437,6 +6533,72 @@ impl Session { None } + /// Bind a Wayland socket that only one application will be given, and hand + /// it to the compositor stamped with that application's identity. + /// + /// Returns the socket's basename, which is what belongs in + /// `WAYLAND_DISPLAY` — a client resolves it under `XDG_RUNTIME_DIR`, so the + /// name sits beside the shared socket rather than in a directory of its own. + /// + /// The socket is bound *here*, before the command is sent and before the + /// application is spawned, so there is no window in which the name exists + /// but nothing is listening on it. The compositor only ever accepts on the + /// fd; the path stays this side, to be unlinked with the instance. + #[cfg(target_os = "linux")] + fn bind_app_socket( + &mut self, + verbose: bool, + event_notify: Arc, + gpu_device: &str, + app_id: &str, + instance_id: &str, + ) -> Option<(String, std::path::PathBuf)> { + use std::os::unix::net::UnixListener; + + // The shared socket's directory is the session's runtime dir, and it is + // also where a client will look for this one. + let shared = self + .ensure_compositor(verbose, event_notify, gpu_device) + .to_string(); + let runtime_dir = std::path::Path::new(&shared).parent()?.to_path_buf(); + // Not "wayland-N": that namespace belongs to bind_auto, and colliding + // with it would be a race against the compositor's own naming. + let name = format!("blit-app-{app_id}-{instance_id}"); + let path = runtime_dir.join(&name); + // A leftover from a crashed instance would make bind fail; the name + // carries a fresh instance id, so anything here is stale by definition. + let _ = std::fs::remove_file(&path); + let listener = match UnixListener::bind(&path) { + Ok(listener) => listener, + Err(e) => { + eprintln!( + "blit-server: cannot bind app socket {}: {e}", + path.display() + ); + return None; + } + }; + let cs = self.compositor.as_ref()?; + let identity = blit_compositor::AppIdentity { + sandbox_engine: "blit".to_string(), + app_id: app_id.to_string(), + instance_id: instance_id.to_string(), + }; + if cs + .handle + .command_tx + .send(blit_compositor::CompositorCommand::AddAppSocket { + fd: std::os::fd::OwnedFd::from(listener), + identity, + }) + .is_err() + { + let _ = std::fs::remove_file(&path); + return None; + } + Some((name, path)) + } + fn live_ptys(&self) -> usize { self.ptys.values().filter(|pty| !pty.exited).count() } @@ -6705,7 +6867,7 @@ impl Session { true } - fn client_list_msg(&self, requester: u64, nonce: u16) -> Vec { + fn client_list_msg(&self, requester: u64, nonce: u16, want_origin: bool) -> Vec { let mut clients: Vec = self .clients .iter() @@ -6758,17 +6920,31 @@ impl Session { terminals, surfaces, subscriptions, + // Built only when asked for: the plain encoder drops it, + // and cloning an origin per client per catalog tick is + // work a watcher that never asked should not pay. + origin: want_origin.then(|| client.origin.wire_origin()), } }) .collect(); clients.sort_unstable_by_key(|entry| entry.client_id); - msg_s2c_client_list(nonce, requester, &clients) + if want_origin { + msg_s2c_client_list2(nonce, requester, &clients) + } else { + msg_s2c_client_list(nonce, requester, &clients) + } } - fn watch_client_list(&mut self, requester: u64, nonce: u16) { - let msg = self.client_list_msg(requester, nonce); + fn watch_client_list(&mut self, requester: u64, nonce: u16, want_origin: bool) { + let msg = self.client_list_msg(requester, nonce, want_origin); if let Some(client) = self.clients.get_mut(&requester) { - client.client_catalog_watches.insert(nonce, msg.clone()); + client.client_catalog_watches.insert( + nonce, + ClientCatalogWatch { + want_origin, + last: msg.clone(), + }, + ); let _ = send_outbox(client, msg); } } @@ -6780,26 +6956,28 @@ impl Session { } fn publish_client_catalogs(&mut self) { - let watches: Vec<(u64, u16)> = self + let watches: Vec<(u64, u16, bool)> = self .clients .iter() .flat_map(|(&client_id, client)| { client .client_catalog_watches - .keys() - .copied() - .map(move |nonce| (client_id, nonce)) + .iter() + .map(move |(&nonce, watch)| (client_id, nonce, watch.want_origin)) }) .collect(); - for (client_id, nonce) in watches { - let msg = self.client_list_msg(client_id, nonce); + for (client_id, nonce, want_origin) in watches { + let msg = self.client_list_msg(client_id, nonce, want_origin); let Some(client) = self.clients.get_mut(&client_id) else { continue; }; - if client.client_catalog_watches.get(&nonce) == Some(&msg) { + let Some(watch) = client.client_catalog_watches.get_mut(&nonce) else { + continue; + }; + if watch.last == msg { continue; } - client.client_catalog_watches.insert(nonce, msg.clone()); + watch.last = msg.clone(); let _ = send_outbox(client, msg); } } @@ -8065,6 +8243,11 @@ struct TerminalScan { /// (docs/protocol.md, "Working directory tracking"): a percent-decoded /// absolute local path of at most `blit_remote::TERM_CWD_MAX` bytes. osc7_cwd: Option, + /// OSC 133/633 semantic-prompt markers in the chunk, in order, each + /// carrying the offset it ended at (docs/design/term-journal.md). Empty + /// for every shell without integration, which is the common case and + /// costs one failed prefix comparison per OSC. + marks: Vec, } /// This machine's hostname, for filtering OSC 7 host components. Cached @@ -8142,6 +8325,7 @@ fn parse_terminal_queries(data: &[u8], size: (u16, u16), cursor: (u16, u16)) -> let mut results = Vec::new(); let mut osc7_cwd = None; + let mut marks = Vec::new(); let mut i = 0; while i < data.len() { if data[i] != 0x1b || i + 1 >= data.len() { @@ -8194,6 +8378,20 @@ fn parse_terminal_queries(data: &[u8], size: (u16, u16), cursor: (u16, u16)) -> osc7_cwd = Some(cwd); } i = end + if data[end] == 0x07 { 1 } else { 2 }; + // OSC 133/633 — shell integration says where each command + // begins and ends (docs/design/term-journal.md). The offset + // matters: a marker means whatever the cursor is when the + // bytes *before* it have been drawn, so the caller replays + // the chunk in segments split here. + if journal::enabled() + && let Some((kind, dialect)) = journal::parse_mark(payload) + { + marks.push(journal::SemanticMark { + kind, + dialect, + at: i, + }); + } continue; } i = end; @@ -8250,9 +8448,335 @@ fn parse_terminal_queries(data: &[u8], size: (u16, u16), cursor: (u16, u16)) -> TerminalScan { responses: results, osc7_cwd, + marks, } } +/// Feed one PTY output chunk to the terminal model: answer the queries in it, +/// note any OSC 7 report, and apply any shell-integration markers. +/// +/// Markers are positional. `OSC 133 ; C` means "output starts *here*", and +/// here is wherever the cursor lands once every byte before the marker has +/// been drawn — so a chunk carrying markers is handed to the driver in +/// segments split at each one. A chunk carrying none, which is every chunk +/// for every shell without integration, takes the single-call path it always +/// took; the only added cost is a failed prefix comparison per OSC. +/// +/// Returns the `S2C_TERM_CWD_EVENT` to broadcast, if the cwd changed. +fn feed_pty_chunk(pty: &mut Pty, id: u16, data: &[u8]) -> Option> { + // A sequence split across a read boundary is invisible to a scan of + // either half, so the unterminated tail of the last chunk goes back in + // front of this one. It has already been drawn, so only the scan sees + // it; the driver still gets `data` and nothing twice. + let carry_len = pty.osc_carry.len(); + let scanned: std::borrow::Cow<[u8]> = if carry_len == 0 { + std::borrow::Cow::Borrowed(data) + } else { + let mut buf = std::mem::take(&mut pty.osc_carry); + buf.extend_from_slice(data); + std::borrow::Cow::Owned(buf) + }; + + let scan = pty::respond_to_queries( + &pty.handle, + &scanned, + pty.driver.size(), + pty.driver.cursor_position(), + ); + + if journal::enabled() { + pty.osc_carry.clear(); + if let Some(tail) = journal::unterminated_osc_tail(&scanned) { + pty.osc_carry.extend_from_slice(&scanned[tail..]); + } + } + + if scan.marks.is_empty() { + pty.driver.process(data); + } else { + let mut fed = 0usize; + for mark in &scan.marks { + let split = mark.at.saturating_sub(carry_len).min(data.len()); + if split > fed { + pty.driver.process(&data[fed..split]); + fed = split; + } + let Pty { + driver, journal, .. + } = pty; + let (cursor_seq, cursor_col) = driver.cursor_seq(); + journal.apply( + mark, + &journal::MarkContext { + cursor_seq, + cursor_col, + read: &|seq, col, end_seq| { + driver + .seq_text(seq, col, Some(end_seq + 1), journal::command_max()) + .text + }, + }, + ); + } + if fed < data.len() { + pty.driver.process(&data[fed..]); + } + } + + note_osc7_cwd(&mut pty.osc7_cwd, id, scan.osc7_cwd) +} + +/// Ceiling on one `S2C_TERM_JOURNAL` listing, well under `MAX_FRAME_SIZE`. +/// The ring bound already caps how many records exist; this caps how much +/// command text a pathological set of them can add up to. +const JOURNAL_REPLY_MAX: usize = 1 << 20; + +/// A client's `max_bytes`, clamped to what the server is willing to build. +fn journal_read_budget(requested: u32) -> usize { + let ceiling = journal::output_max(); + if requested == 0 { + ceiling + } else { + (requested as usize).min(ceiling) + } +} + +/// `S2C_TERM_OUTPUT` flags for a completed read. +fn output_flags(read: &SeqText, alt_screen: bool) -> u8 { + let mut flags = 0; + if read.truncated { + flags |= blit_remote::journal::OUTPUT_TRUNCATED; + } + if read.evicted { + flags |= blit_remote::journal::OUTPUT_EVICTED; + } + if alt_screen { + flags |= blit_remote::journal::OUTPUT_ALT_SCREEN; + } + flags +} + +/// Answer `C2S_TERM_JOURNAL`: the command list, newest-relative or absolute. +fn journal_list_reply(sess: &Session, req: &blit_remote::journal::JournalRequest) -> Vec { + use blit_remote::journal as j; + let Some(pty) = sess.ptys.get(&req.pty_id) else { + return j::msg_s2c_term_journal(req.nonce, req.pty_id, STATUS_NOT_FOUND, 0, 0, &[]); + }; + let (cursor_seq, _) = pty.driver.cursor_seq(); + let oldest_seq = pty.driver.oldest_seq(); + let indices: Vec = pty.journal.iter().map(|r| r.index).collect(); + let limit = if req.limit == 0 { + indices.len() + } else { + req.limit as usize + }; + + let chosen: Vec = if req.flags & j::JOURNAL_TAIL != 0 { + // Counting back from the newest: skip `from_index` of them, then + // take `limit` — so `from_index = 0` is "the last `limit`". + let end = indices.len().saturating_sub(req.from_index as usize); + let start = end.saturating_sub(limit); + indices[start..end].to_vec() + } else { + indices + .into_iter() + .filter(|&i| i >= req.from_index) + .take(limit) + .collect() + }; + + let mut records = Vec::with_capacity(chosen.len()); + let mut bytes = 0usize; + for index in chosen { + let Some(record) = pty.journal.snapshot(index, cursor_seq, oldest_seq) else { + continue; + }; + bytes += record.command.len() + 64; + if bytes > JOURNAL_REPLY_MAX { + break; + } + records.push(record); + } + j::msg_s2c_term_journal( + req.nonce, + req.pty_id, + STATUS_OK, + pty.journal.oldest_index(), + pty.journal.next_index(), + &records, + ) +} + +/// Answer `C2S_TERM_OUTPUT`: one command's output region. +fn journal_output_reply(sess: &Session, req: &blit_remote::journal::OutputRequest) -> Vec { + use blit_remote::journal as j; + let not_found = + || j::msg_s2c_term_output(req.nonce, req.pty_id, STATUS_NOT_FOUND, 0, 0, 0, 0, 0, ""); + let Some(pty) = sess.ptys.get(&req.pty_id) else { + return not_found(); + }; + let (cursor_seq, _) = pty.driver.cursor_seq(); + let oldest_seq = pty.driver.oldest_seq(); + let index = if req.index == j::JOURNAL_INDEX_LATEST { + pty.journal.latest().map(|r| r.index) + } else { + Some(req.index) + }; + let Some(record) = index.and_then(|i| pty.journal.snapshot(i, cursor_seq, oldest_seq)) else { + return not_found(); + }; + // A running command has no end yet, so its output is read to wherever + // the terminal has got to; a finished one is frozen at its `D` marker. + let end = (!record.running()).then_some(record.end_seq); + let read = pty + .driver + .seq_text(record.start_seq, 0, end, journal_read_budget(req.max_bytes)); + j::msg_s2c_term_output( + req.nonce, + req.pty_id, + STATUS_OK, + output_flags(&read, pty.driver.alt_screen()), + read.start_seq, + read.start_col, + read.next_seq, + read.next_col, + &read.text, + ) +} + +/// Answer `C2S_TERM_SINCE`: everything appended since the client's cursor. +fn term_since_reply(sess: &Session, req: &blit_remote::journal::SinceRequest) -> Vec { + use blit_remote::journal as j; + let Some(pty) = sess.ptys.get(&req.pty_id) else { + return j::msg_s2c_term_output(req.nonce, req.pty_id, STATUS_NOT_FOUND, 0, 0, 0, 0, 0, ""); + }; + let alt_screen = pty.driver.alt_screen(); + // A probe establishes a starting cursor without dragging back everything + // already on screen — which is what makes "wait for output after now" + // expressible at all. + if req.flags & j::SINCE_PROBE != 0 { + let (seq, col) = pty.driver.cursor_seq(); + return j::msg_s2c_term_output( + req.nonce, + req.pty_id, + STATUS_OK, + output_flags(&SeqText::default(), alt_screen), + seq, + col, + seq, + col, + "", + ); + } + let read = pty.driver.seq_text( + req.from_seq, + req.from_col, + None, + journal_read_budget(req.max_bytes), + ); + j::msg_s2c_term_output( + req.nonce, + req.pty_id, + STATUS_OK, + output_flags(&read, alt_screen), + read.start_seq, + read.start_col, + read.next_seq, + read.next_col, + &read.text, + ) +} + +/// Register a `C2S_TERM_JOURNAL_WAIT`, answering straight away when the +/// command it names has already finished. +fn arm_journal_waiter( + sess: &mut Session, + client_id: u64, + req: &blit_remote::journal::WaitRequest, + now: Instant, +) { + use blit_remote::journal as j; + let reply = |sess: &Session, status: u8, record: &blit_remote::journal::CommandRecord| { + if let Some(client) = sess.clients.get(&client_id) { + let _ = send_outbox( + client, + j::msg_s2c_term_command(req.nonce, req.pty_id, status, record), + ); + } + }; + let empty = blit_remote::journal::CommandRecord::default(); + let in_flight = sess + .journal_waiters + .iter() + .filter(|w| w.client_id == client_id) + .count(); + if in_flight >= journal::MAX_WAITERS_PER_CLIENT { + reply(sess, STATUS_BUDGET, &empty); + return; + } + let timeout = req.timeout_ms.min(journal::WAIT_TIMEOUT_MAX_MS); + let mut waiter = journal::Waiter { + client_id, + nonce: req.nonce, + pty_id: req.pty_id, + index: (req.index != j::JOURNAL_INDEX_LATEST).then_some(req.index), + deadline: now + Duration::from_millis(timeout as u64), + }; + let pty = sess.ptys.get(&req.pty_id); + let cursor_seq = pty.map(|p| p.driver.cursor_seq().0).unwrap_or(0); + let oldest_seq = pty.map(|p| p.driver.oldest_seq()).unwrap_or(0); + match waiter.poll(pty.map(|p| &p.journal), cursor_seq, oldest_seq, now) { + journal::WaitOutcome::Ready(record) => reply(sess, STATUS_OK, &record), + journal::WaitOutcome::Gone => reply(sess, STATUS_NOT_FOUND, &empty), + journal::WaitOutcome::Pending => sess.journal_waiters.push(waiter), + } +} + +/// Answer every journal waiter whose command has finished or timed out. +/// +/// Called from the tick, which is where a `D` marker lands, and from the +/// supervisor, which is what wakes for a timeout on an otherwise idle +/// terminal. +fn resolve_journal_waiters(sess: &mut Session, now: Instant) { + if sess.journal_waiters.is_empty() { + return; + } + let Session { + ptys, + clients, + journal_waiters, + .. + } = sess; + journal_waiters.retain_mut(|waiter| { + // A client that hung up takes its waiters with it. + let Some(client) = clients.get(&waiter.client_id) else { + return false; + }; + let pty = ptys.get(&waiter.pty_id); + let cursor_seq = pty.map(|p| p.driver.cursor_seq().0).unwrap_or(0); + let oldest_seq = pty.map(|p| p.driver.oldest_seq()).unwrap_or(0); + let (status, record) = + match waiter.poll(pty.map(|p| &p.journal), cursor_seq, oldest_seq, now) { + journal::WaitOutcome::Pending => return true, + journal::WaitOutcome::Ready(record) => (STATUS_OK, record), + journal::WaitOutcome::Gone => ( + STATUS_NOT_FOUND, + blit_remote::journal::CommandRecord::default(), + ), + }; + let _ = send_outbox( + client, + blit_remote::journal::msg_s2c_term_command( + waiter.nonce, + waiter.pty_id, + status, + &record, + ), + ); + false + }); +} + /// Record an OSC 7 report against a PTY's stored cwd; returns the /// `S2C_TERM_CWD_EVENT` to broadcast only when the value changed. Shells /// re-emit OSC 7 at every prompt, so identical repeats must produce no @@ -8463,7 +8987,8 @@ async fn supervisor_loop(state: AppState) { /// The soonest instant the supervisor has work to do, or `None` when nothing /// is armed. fn earliest_armed_deadline(sess: &Session) -> Option { - sess.ptys + let ptys = sess + .ptys .values() .filter(|pty| !pty.exited) .filter_map(|pty| { @@ -8473,7 +8998,14 @@ fn earliest_armed_deadline(sess: &Session) -> Option { .chain(pty.exit_drain_deadline) .min() }) - .min() + .min(); + // A journal wait is the other thing with a clock on it: without this the + // supervisor would sleep through a timeout on an idle terminal. + let waits = sess.journal_waiters.iter().map(|w| w.deadline).min(); + match (ptys, waits) { + (Some(a), Some(b)) => Some(a.min(b)), + (a, b) => a.or(b), + } } /// Terminals to evict to stay inside the retention bounds, oldest first. @@ -8586,6 +9118,9 @@ async fn supervise(state: &AppState) { let now = Instant::now(); let fallback_due = { let mut sess = state.session.lock().await; + // The tick answers waiters whose command finished; nothing else wakes + // for one that simply ran out of time on a quiet terminal. + resolve_journal_waiters(&mut sess, now); for pty in sess.ptys.values_mut().filter(|pty| !pty.exited) { if pty.exit_drain_deadline.is_none() && pty::poll_child_exited(&pty.handle) { pty.exit_drain_deadline = Some(now + PTY_EXIT_DRAIN_GRACE); @@ -8822,8 +9357,14 @@ async fn cleanup_pty_internal(pty_id: u16, generation: Option, state: &AppS pty::close_pty(&pty.handle); pty.exit_status = pty::collect_exit_status(&pty.handle); pty.mark_dirty(); + // A command still running when the shell dies never gets its `D` + // marker; closing it here is what stops a waiter hanging until its + // timeout for output that is never coming. + let end_seq = pty.driver.cursor_seq().0; + pty.journal.note_pty_exit(end_seq); let msg = blit_remote::msg_exited_reason(pty_id, pty.exit_status, pty.exit_reason); sess.send_to_all(&msg); + resolve_journal_waiters(&mut sess, Instant::now()); } } @@ -9274,6 +9815,10 @@ async fn tick(state: &AppState) -> TickOutcome { CachedSurfaceInfo { surface_id, parent_id, + // Filled by the SurfaceOrigin that follows + // immediately when the client is on a per-app + // socket; stays None for the shared one. + origin: None, width, height, logical_width: 0, @@ -9450,6 +9995,29 @@ async fn tick(state: &AppState) -> TickOutcome { } broadcast.push(msg_surface_app_id(surface_id, &app_id)); } + CompositorEvent::SurfaceOrigin { + surface_id, + sandbox_engine, + app_id, + instance_id, + } => { + // Cached so a client attaching later learns it too: the + // compositor sends this once, at creation, and identity + // cannot change while the connection lives. + if let Some(info) = cs.surfaces.get_mut(&surface_id) { + info.origin = Some(SurfaceOrigin { + sandbox_engine: sandbox_engine.clone(), + app_id: app_id.clone(), + instance_id: instance_id.clone(), + }); + } + broadcast.push(msg_surface_origin( + surface_id, + &sandbox_engine, + &app_id, + &instance_id, + )); + } CompositorEvent::SurfaceActivated { surface_id } => { // Nothing to cache — the client is asked to raise and // focus the pane, which it answers with C2S_SURFACE_FOCUS. @@ -12499,31 +13067,17 @@ async fn tick(state: &AppState) -> TickOutcome { match input { PtyInput::Data(data) => { budget = budget.saturating_sub(data.len()); - let osc7 = pty::respond_to_queries( - &pty.handle, - &data, - pty.driver.size(), - pty.driver.cursor_position(), - ); - if let Some(msg) = note_osc7_cwd(&mut pty.osc7_cwd, id, osc7) { + if let Some(msg) = feed_pty_chunk(pty, id, &data) { cwd_msgs.push(msg); } - pty.driver.process(&data); pty.mark_output_dirty(now, output_coalesce_cap); } PtyInput::SyncBoundary { before } => { budget = budget.saturating_sub(before.len()); if !before.is_empty() { - let osc7 = pty::respond_to_queries( - &pty.handle, - &before, - pty.driver.size(), - pty.driver.cursor_position(), - ); - if let Some(msg) = note_osc7_cwd(&mut pty.osc7_cwd, id, osc7) { + if let Some(msg) = feed_pty_chunk(pty, id, &before) { cwd_msgs.push(msg); } - pty.driver.process(&before); pty.mark_output_dirty(now, output_coalesce_cap); } if !pty.driver.synced_output() { @@ -12548,6 +13102,9 @@ async fn tick(state: &AppState) -> TickOutcome { for msg in cwd_msgs { sess.send_to_all(&msg); } + // A `D` marker in the bytes just processed is what finishes a command, + // so this is the first moment a waiter can be answered. + resolve_journal_waiters(&mut sess, now); if parse_budget_hit { // Leftover output is already queued, so re-tick right after this // round instead of waiting on the reader's notify — the permit for @@ -13305,6 +13862,10 @@ struct FsSyncs { /// cap, its own set so search and index nonce spaces stay independent. inflight_searches: std::sync::Arc>>, inflight_greps: std::sync::Arc>>, + /// Nonces of `FS_READ` batches in flight — same discipline again: the read + /// happens off-thread, so without a cap a client could queue as many as it + /// can send. + inflight_reads: std::sync::Arc>>, /// Live chunked uploads on this connection (upload_id → sync_id routing /// plus the concurrent-upload cap). Shared with the sync sinks, which /// free the id of a BEGIN the engine refused. @@ -13398,6 +13959,25 @@ impl FsSyncs { ))) } + /// Reserve a nonce for an in-flight `FS_READ`, same cap and failure split + /// as [`FsSyncs::reserve_index`]: a read is off-thread I/O a client can + /// otherwise stack up. + fn reserve_read(&self, nonce: u16) -> Result, u8> { + use blit_remote::fs::{FS_DONE_BUDGET, FS_DONE_INVALID}; + let mut set = self.inflight_reads.lock().unwrap(); + if set.contains(&nonce) { + return Err(FS_DONE_INVALID); + } + if set.len() >= fs_walk_inflight() { + return Err(FS_DONE_BUDGET); + } + set.insert(nonce); + Ok(std::sync::Arc::new(blit_fssync::InflightGuard::new( + self.inflight_reads.clone(), + nonce, + ))) + } + /// Reserve a nonce for an in-flight `FS_GREP` walk, same cap and /// failure split as [`FsSyncs::reserve_index`]. fn reserve_grep(&self, nonce: u16) -> Result, u8> { @@ -13884,10 +14464,72 @@ const FS_INDEX_MAX_BYTES: usize = 48 * 1024 * 1024; /// with a bare `*` — the dotfiles-repo-at-$HOME pattern — blanks every /// non-repo subtree) falls back to an ignore-free walk: an empty index /// would otherwise read as "no files here" and never consult the server. +/// Classify one path for `FS_READ` without reading it. +fn stat_one_for_fs_read(path: &str, per_file: u64) -> u8 { + use blit_remote::fs::{ + FS_FILE_NOT_FOUND, FS_FILE_OK, FS_FILE_OTHER, FS_FILE_TOO_LARGE, FS_FILE_UNREADABLE, + }; + match std::fs::metadata(path) { + Err(error) => match error.kind() { + std::io::ErrorKind::NotFound => FS_FILE_NOT_FOUND, + std::io::ErrorKind::PermissionDenied => FS_FILE_UNREADABLE, + _ => FS_FILE_OTHER, + }, + Ok(meta) if !meta.is_file() => FS_FILE_UNREADABLE, + Ok(meta) if meta.len() > per_file => FS_FILE_TOO_LARGE, + Ok(_) => FS_FILE_OK, + } +} + +/// Read one path for `FS_READ`: its record status and content. +/// +/// `left` is what remains of the reply budget, so a file that would not fit is +/// reported the same way one over the caller's own ceiling is — the caller is not +/// getting it either way, and the distinction is not worth a status. +fn read_one_for_fs_read(path: &str, per_file: u64, left: usize) -> (u8, Vec) { + use blit_remote::fs::{ + FS_FILE_NOT_FOUND, FS_FILE_OK, FS_FILE_OTHER, FS_FILE_TOO_LARGE, FS_FILE_UNREADABLE, + }; + let kind = |error: &std::io::Error| match error.kind() { + std::io::ErrorKind::NotFound => FS_FILE_NOT_FOUND, + std::io::ErrorKind::PermissionDenied => FS_FILE_UNREADABLE, + _ => FS_FILE_OTHER, + }; + match std::fs::metadata(path) { + Err(error) => (kind(&error), Vec::new()), + // A directory has no content to answer with, and `FS_INDEX` is the + // message that describes one. + Ok(meta) if !meta.is_file() => (FS_FILE_UNREADABLE, Vec::new()), + Ok(meta) if meta.len() > per_file || meta.len() as usize > left => { + (FS_FILE_TOO_LARGE, Vec::new()) + } + Ok(_) => match std::fs::read(path) { + // It grew between the stat and the read. + Ok(body) if body.len() as u64 > per_file => (FS_FILE_TOO_LARGE, Vec::new()), + Ok(body) => (FS_FILE_OK, body), + Err(error) => (kind(&error), Vec::new()), + }, + } +} + fn fs_index_walk(root: &std::path::Path, max_entries: usize) -> (Vec, bool) { - let (paths, truncated) = fs_index_walk_inner(root, max_entries, true); - if paths.is_empty() && !truncated { - return fs_index_walk_inner(root, max_entries, false); + fs_index_walk_kind(root, max_entries, false, false) +} + +/// [`fs_index_walk`], listing directories instead of files when `dirs_only`. +/// +/// The ignore-free retry is skipped for directories: an ignore rule that blanks +/// every file in a tree still leaves its directories, so an empty answer there +/// means an empty tree and the second walk would only cost time. +fn fs_index_walk_kind( + root: &std::path::Path, + max_entries: usize, + dirs_only: bool, + follow: bool, +) -> (Vec, bool) { + let (paths, truncated) = fs_index_walk_inner(root, max_entries, true, dirs_only, follow); + if paths.is_empty() && !truncated && !dirs_only { + return fs_index_walk_inner(root, max_entries, false, dirs_only, follow); } (paths, truncated) } @@ -13896,6 +14538,8 @@ fn fs_index_walk_inner( root: &std::path::Path, max_entries: usize, use_ignores: bool, + dirs_only: bool, + follow: bool, ) -> (Vec, bool) { let mut paths: Vec = Vec::new(); let mut bytes = 0usize; @@ -13909,7 +14553,7 @@ fn fs_index_walk_inner( let mut builder = ignore::WalkBuilder::new(root); builder .hidden(false) - .follow_links(false) + .follow_links(follow) .filter_entry(|e| e.file_name() != std::ffi::OsStr::new(".git")); if !use_ignores { builder.standard_filters(false); @@ -13920,14 +14564,27 @@ fn fs_index_walk_inner( truncated = true; break; } - if !entry.file_type().is_some_and(|ft| ft.is_file()) { + // A symlink counts as whatever it points at, which costs one stat and is + // the difference between seeing a Nix system directory and not: every + // `.desktop` file and every icon directory under + // `/run/current-system/sw/share` is a link into the store, and the walk + // does not follow links, so `file_type` calls them all symlinks. + let wanted = match entry.file_type() { + Some(kind) if kind.is_dir() => dirs_only, + Some(kind) if kind.is_file() => !dirs_only, + Some(kind) if kind.is_symlink() => { + std::fs::metadata(entry.path()).is_ok_and(|target| target.is_dir() == dirs_only) + } + _ => false, + }; + if !wanted { continue; } let Ok(rel) = entry.path().strip_prefix(root) else { continue; }; if rel.as_os_str().is_empty() { - continue; // the root itself, when it is a file + continue; // the root itself, which the caller already has } // Truncation is exact: it fires only when a file would be dropped, // never on a trailing directory after the last counted file. @@ -14078,8 +14735,8 @@ async fn handle_fs_message_with_jobs( ) { use blit_fssync::{OpReq, UploadBeginReq, WriteReq}; use blit_remote::fs::{ - C2S_FS_ACK, C2S_FS_FETCH, C2S_FS_GREP, C2S_FS_INDEX, C2S_FS_OP, C2S_FS_SEARCH, C2S_FS_STOP, - C2S_FS_SYNC, C2S_FS_UPLOAD_BEGIN, C2S_FS_UPLOAD_CANCEL, C2S_FS_UPLOAD_CHUNK, + C2S_FS_ACK, C2S_FS_FETCH, C2S_FS_GREP, C2S_FS_INDEX, C2S_FS_OP, C2S_FS_READ, C2S_FS_SEARCH, + C2S_FS_STOP, C2S_FS_SYNC, C2S_FS_UPLOAD_BEGIN, C2S_FS_UPLOAD_CANCEL, C2S_FS_UPLOAD_CHUNK, C2S_FS_UPLOAD_FINISH, C2S_FS_WRITE, FS_DONE_BUDGET, FS_DONE_INVALID, FS_DONE_NOT_FOUND, FS_DONE_OK, FS_DONE_OTHER, FS_DONE_PERMISSION, FS_DONE_UNKNOWN_UPLOAD, FS_DONE_WRONG_TYPE, FS_FILE_OTHER, FS_INDEX_TRUNCATED, FS_STATUS_OK, FS_STATUS_OTHER, FS_STATUS_RESOURCE_LIMIT, @@ -14088,8 +14745,8 @@ async fn handle_fs_message_with_jobs( FS_SYNC_RECURSIVE, FS_SYNC_SINGLE, fs_sync_flags_valid, msg_fs_done, msg_fs_file, msg_fs_index_result, msg_fs_search_result, msg_fs_synced, msg_fs_upload_begin_result, msg_fs_upload_chunk_result, msg_fs_upload_finish_result, parse_fs_index, parse_fs_op, - parse_fs_search, parse_fs_upload_begin, parse_fs_upload_chunk, parse_fs_upload_finish, - parse_fs_write, + parse_fs_read, parse_fs_search, parse_fs_upload_begin, parse_fs_upload_chunk, + parse_fs_upload_finish, parse_fs_write, }; match data[0] { C2S_FS_SEARCH => { @@ -14222,15 +14879,91 @@ async fn handle_fs_message_with_jobs( } } } + C2S_FS_READ => { + // One-shot read of a fixed set of paths (docs/design/fs-read.md) — + // no sync, nothing watched. Off-thread and capped by `reserve_read` + // like every other walk in this family. + if let Some((nonce, flags, max_bytes, groups)) = parse_fs_read(data) { + use blit_remote::fs::{ + FS_FILE_NOT_FOUND, FS_FILE_OK, FS_READ_DEFAULT_BYTES, FS_READ_FIRST, + FS_READ_FLAGS_KNOWN, FS_READ_MAX_TOTAL_BYTES, FS_READ_NO_CONTENT, + msg_fs_read_result, + }; + if flags & !FS_READ_FLAGS_KNOWN != 0 { + let _ = out.send(msg_fs_read_result(nonce, FS_DONE_INVALID, &[])); + return; + } + let guard = match syncs.reserve_read(nonce) { + Ok(guard) => guard, + Err(status) => { + let _ = out.send(msg_fs_read_result(nonce, status, &[])); + return; + } + }; + let out = out.clone(); + let work = move || { + let _guard = guard; + let per_file = u64::from(if max_bytes == 0 { + FS_READ_DEFAULT_BYTES + } else { + max_bytes + }); + let first_only = flags & FS_READ_FIRST != 0; + let no_content = flags & FS_READ_NO_CONTENT != 0; + let mut left = FS_READ_MAX_TOTAL_BYTES; + let mut records: Vec<(u8, String, Vec)> = Vec::new(); + for group in groups { + let mut answered = false; + for path in group { + let (status, body) = if no_content { + (stat_one_for_fs_read(&path, per_file), Vec::new()) + } else { + read_one_for_fs_read(&path, per_file, left) + }; + left = left.saturating_sub(body.len()); + // FIRST asks for the first path it can *have*, so a + // miss, an unreadable file and an oversized one are + // all stepped over rather than answered. + if first_only { + if status == FS_FILE_OK { + records.push((status, path, body)); + answered = true; + break; + } + continue; + } + records.push((status, path, body)); + } + // One record per group either way, so a caller can align + // answers with questions by position. + if first_only && !answered { + records.push((FS_FILE_NOT_FOUND, String::new(), Vec::new())); + } + } + let borrowed: Vec<(u8, &str, &[u8])> = records + .iter() + .map(|(status, path, body)| (*status, path.as_str(), body.as_slice())) + .collect(); + let _ = out.send(msg_fs_read_result(nonce, FS_DONE_OK, &borrowed)); + }; + if let Some(jobs) = jobs { + let _ = jobs.spawn_blocking(data.len(), work); + } else { + std::thread::spawn(work); + } + } + } C2S_FS_INDEX => { // Candidate list for client-side fuzzy search // (docs/design/fs-search.md) — no sync. Walk off-thread, capped // by `reserve_index` so a client can't stack up walks. if let Some((nonce, flags, root)) = parse_fs_index(data) { - if flags != 0 { + if flags & !blit_remote::fs::FS_INDEX_FLAGS_KNOWN != 0 { let _ = out.send(msg_fs_index_result(nonce, FS_DONE_INVALID, 0, &[])); return; } + let dirs_only = flags & blit_remote::fs::FS_INDEX_DIRS_ONLY != 0; + let follow = flags & blit_remote::fs::FS_INDEX_FOLLOW_LINKS != 0; let guard = match syncs.reserve_index(nonce) { Ok(guard) => guard, Err(status) => { @@ -14257,8 +14990,12 @@ async fn handle_fs_message_with_jobs( Ok(canon) => match std::fs::read_dir(&canon) { Err(err) => msg_fs_index_result(nonce, io_status(&err), 0, &[]), Ok(_) => { - let (paths, truncated) = - fs_index_walk(&canon, fs_index_max_entries()); + let (paths, truncated) = fs_index_walk_kind( + &canon, + fs_index_max_entries(), + dirs_only, + follow, + ); let flags = if truncated { FS_INDEX_TRUNCATED } else { 0 }; msg_fs_index_result(nonce, FS_DONE_OK, flags, &paths) } @@ -15721,6 +16458,38 @@ async fn handle_git_message( }, ); } + C2S_GIT_WORKTREES => { + let Some(req) = parse_git_worktrees(data) else { + if let Some(n) = git_nonce(data) { + let _ = out.send(msg_git_worktrees_resp(n, GIT_STATUS_INVALID, 0, &[])); + } + return; + }; + let nonce = req.nonce; + let Some(entry) = repos.map.get(&req.repo_id) else { + let _ = out.send(msg_git_worktrees_resp(nonce, GIT_STATUS_UNKNOWN_ID, 0, &[])); + return; + }; + let handle = entry.handle.clone(); + let owned = (req.flags, req.after_pos); + git_request( + repos, + nonce, + out, + move |status| msg_git_worktrees_resp(nonce, status, 0, &[]), + move |cancel| { + handle.worktrees( + &GitWorktreesRequest { + nonce, + repo_id: 0, + flags: owned.0, + after_pos: owned.1, + }, + &cancel, + ) + }, + ); + } _ => {} } } @@ -16626,6 +17395,11 @@ async fn handle_client_registered - { - let sess = state.session.lock().await; - sess.ptys - .get(&request.src_pty_id) - .and_then(|pty| pty::pty_cwd(&pty.handle)) - .map(String::into_bytes) - } - _ => None, - }; - processes.spawn(&data, pty_cwd.as_deref()); + // Both the source terminal's cwd and the session + // environment live behind the session lock, which the + // process family itself never holds — so resolve them + // here, as owned data, and take the lock at most once. + let (pty_cwd, session_env) = + match blit_remote::process::parse_process_spawn(&data) { + Ok(request) => { + let from_pty = + request.cwd_kind == blit_remote::process::PROCESS_CWD_FROM_PTY; + let wants_session_env = request.flags + & blit_remote::process::PROCESS_SPAWN_SESSION_ENV + != 0; + if from_pty || wants_session_env { + let mut sess = state.session.lock().await; + let cwd = from_pty + .then(|| { + sess.ptys + .get(&request.src_pty_id) + .and_then(|pty| pty::pty_cwd(&pty.handle)) + .map(String::into_bytes) + }) + .flatten(); + let env = wants_session_env.then(|| { + // A supervised app can be the first + // thing in the session, so the + // compositor may not exist yet. Asking + // for the session environment is what + // brings it up. + let socket_name = sess + .ensure_compositor( + config.verbose, + notify_for_compositor.clone(), + &config.vaapi_device, + ) + .to_string(); + #[cfg(target_os = "linux")] + let pulse_server = sess.pulse_server_path(); + #[cfg(not(target_os = "linux"))] + let pulse_server: Option = None; + #[cfg(target_os = "linux")] + let pipewire_remote = sess.pipewire_remote_path(); + #[cfg(not(target_os = "linux"))] + let pipewire_remote: Option = None; + app_env::session_env( + Some(&socket_name), + sess.x_display().as_deref(), + sess.desktop_bus_address().as_deref(), + pulse_server.as_deref(), + pipewire_remote.as_deref(), + ) + }); + (cwd, env) + } else { + (None, None) + } + } + Err(_) => (None, None), + }; + processes.spawn(&data, pty_cwd.as_deref(), session_env); } else { processes.handle(&data); } @@ -17494,6 +18347,20 @@ async fn handle_client_registered blit_remote::msg_app_socket_reply( + nonce, + blit_remote::STATUS_OK, + &name, + ), + None => blit_remote::msg_app_socket_reply( + nonce, + blit_remote::STATUS_OTHER, + "", + ), + } + } + }; + #[cfg(not(target_os = "linux"))] + let reply = + blit_remote::msg_app_socket_reply(nonce, blit_remote::STATUS_WRONG_TYPE, ""); + let _ = fs_out.send(reply); + } + continue; + } + + // The server's own environment (docs/design/env.md). One request, one + // reply, no state — but it hands over every credential the server was + // started with, so BLIT_ENV=0 refuses it with PERMISSION. + if blit_remote::env::is_c2s_env(data[0]) { + if let Ok(nonce) = blit_remote::env::parse_env_get(&data) { + let reply = if env_enabled { + let entries = std::env::vars_os() + .map(|(key, value)| { + #[cfg(unix)] + { + use std::os::unix::ffi::OsStringExt; + (key.into_vec(), value.into_vec()) + } + #[cfg(not(unix))] + { + ( + key.to_string_lossy().into_owned().into_bytes(), + value.to_string_lossy().into_owned().into_bytes(), + ) + } + }) + .collect(); + blit_remote::env::msg_env(nonce, STATUS_OK, &entries) + } else { + blit_remote::env::msg_env( + nonce, + blit_remote::STATUS_PERMISSION, + &std::collections::BTreeMap::new(), + ) + }; + // An environment big enough to refuse still owes the caller an + // answer under its nonce, or it waits forever. + let reply = reply.unwrap_or_else(|error| { + blit_remote::env::msg_env( + nonce, + error.status(), + &std::collections::BTreeMap::new(), + ) + .expect("an empty reply always encodes") + }); + let _ = fs_out.send(reply); + } + continue; + } + // Server KV store (docs/design/kv.md): connection-scoped // subscriptions over one process-global store, never the session // mutex. BLIT_KV=0 refuses nonce-bearing requests with PERMISSION. @@ -18046,7 +19017,20 @@ async fn handle_client_registered= 3 { let nonce = u16::from_le_bytes([data[1], data[2]]); let sess = state.session.lock().await; @@ -18068,10 +19052,13 @@ async fn handle_client_registered { if let Some(client) = sess.clients.get(&client_id) { - let _ = send_outbox(client, sess.client_list_msg(client_id, nonce)); + let _ = send_outbox( + client, + sess.client_list_msg(client_id, nonce, want_origin), + ); } } - C2S_CLIENT_WATCH => sess.watch_client_list(client_id, nonce), + C2S_CLIENT_WATCH => sess.watch_client_list(client_id, nonce, want_origin), _ => sess.unwatch_client_list(client_id, nonce), } continue; @@ -20169,6 +21156,12 @@ async fn handle_client_registered { + if let Some(req) = blit_remote::journal::parse_term_journal(&data) { + let reply = journal_list_reply(&sess, &req); + if let Some(client) = sess.clients.get(&client_id) { + let _ = send_outbox(client, reply); + } + } + } + blit_remote::journal::C2S_TERM_OUTPUT => { + if let Some(req) = blit_remote::journal::parse_term_output(&data) { + let reply = journal_output_reply(&sess, &req); + if let Some(client) = sess.clients.get(&client_id) { + let _ = send_outbox(client, reply); + } + } + } + blit_remote::journal::C2S_TERM_SINCE => { + if let Some(req) = blit_remote::journal::parse_term_since(&data) { + let reply = term_since_reply(&sess, &req); + if let Some(client) = sess.clients.get(&client_id) { + let _ = send_outbox(client, reply); + } + } + } + blit_remote::journal::C2S_TERM_JOURNAL_WAIT => { + if let Some(req) = blit_remote::journal::parse_term_journal_wait(&data) { + arm_journal_waiter(&mut sess, client_id, &req, Instant::now()); + need_nudge = true; + } + } C2S_COPY_RANGE if data.len() >= 18 => { let nonce = u16::from_le_bytes([data[1], data[2]]); let pid = u16::from_le_bytes([data[3], data[4]]); @@ -20410,7 +21433,17 @@ async fn handle_client_registered) { + /// `session_env` is `Some` only when the request carried + /// `PROCESS_SPAWN_SESSION_ENV` and the session had an environment to give. + /// It is resolved by the caller because the session lives behind an async + /// mutex this family never holds. + pub(crate) fn spawn( + &self, + data: &[u8], + pty_cwd: Option<&[u8]>, + session_env: Option, + ) { let nonce = read_u16(data, 1); let process_id = read_u32(data, 3); let req = match parse_process_spawn(data) { @@ -987,11 +996,19 @@ impl Manager { let pty_cwd = pty_cwd.map(ToOwned::to_owned); let manager = self.clone(); tokio::spawn(async move { - manager.run_spawn(pending, request, pty_cwd).await; + manager + .run_spawn(pending, request, pty_cwd, session_env) + .await; }); } - async fn run_spawn(&self, pending: Arc, request: Vec, pty_cwd: Option>) { + async fn run_spawn( + &self, + pending: Arc, + request: Vec, + pty_cwd: Option>, + session_env: Option, + ) { let permit = tokio::select! { permit = self.server.0.spawn_slots.acquire() => permit.ok(), _ = pending.cancel.notified() => None, @@ -1026,7 +1043,7 @@ impl Manager { ); return; } - let mut command = command_for(&req, pty_cwd.as_deref()); + let mut command = command_for(&req, pty_cwd.as_deref(), session_env.as_ref()); let merged_reader = if merged { match configure_merged_output(&mut command) { Ok(reader) => Some(reader), @@ -2298,9 +2315,23 @@ fn watched_error( } #[cfg(unix)] -fn command_for(req: &ProcessSpawnRequest<'_>, pty_cwd: Option<&[u8]>) -> Command { +fn command_for( + req: &ProcessSpawnRequest<'_>, + pty_cwd: Option<&[u8]>, + session_env: Option<&crate::app_env::SessionEnv>, +) -> Command { let mut command = Command::new(OsStr::from_bytes(req.argv[0])); command.args(req.argv[1..].iter().map(|arg| OsStr::from_bytes(arg))); + // The session environment goes on first so the client's own entries still + // win, matching the documented "explicit entries replace inherited ones". + if let Some(session) = session_env { + for key in &session.remove { + command.env_remove(key); + } + for (key, value) in &session.set { + command.env(key, value); + } + } for (key, value) in &req.env { command.env( OsString::from_vec(key.to_vec()), @@ -2381,7 +2412,11 @@ fn command_for(req: &ProcessSpawnRequest<'_>, pty_cwd: Option<&[u8]>) -> Command } #[cfg(windows)] -fn command_for(req: &ProcessSpawnRequest<'_>, pty_cwd: Option<&[u8]>) -> Command { +fn command_for( + req: &ProcessSpawnRequest<'_>, + pty_cwd: Option<&[u8]>, + session_env: Option<&crate::app_env::SessionEnv>, +) -> Command { let argv = req .argv .iter() @@ -2389,6 +2424,16 @@ fn command_for(req: &ProcessSpawnRequest<'_>, pty_cwd: Option<&[u8]>) -> Command .collect::>(); let mut command = Command::new(argv[0]); command.args(&argv[1..]); + // There is no Wayland session to join on Windows, so the resolver hands back + // nothing; the parameter exists to keep one signature across platforms. + if let Some(session) = session_env { + for key in &session.remove { + command.env_remove(key); + } + for (key, value) in &session.set { + command.env(key, value); + } + } for (key, value) in &req.env { command.env( std::str::from_utf8(key).expect("Windows env key validated UTF-8"), @@ -3471,6 +3516,7 @@ mod tests { vec![b"/bin/sh", b"-c", b"printf 'a\\000b'; printf err >&2"], ), None, + None, ); assert_eq!(await_started(&mut rx).await.status, STATUS_OK); let mut stdout = Vec::new(); @@ -3526,6 +3572,7 @@ mod tests { ], ), None, + None, ); assert_eq!(await_started(&mut rx).await.status, STATUS_OK); let mut received = 0u64; @@ -3561,8 +3608,8 @@ mod tests { let (out, mut rx) = outbox(&server); let manager = server.endpoint(out); let msg = spawn_message(9, 0, vec![b"true"]); - manager.spawn(&msg, None); - manager.spawn(&msg, None); + manager.spawn(&msg, None, None); + manager.spawn(&msg, None, None); let reply_msg = recv(&mut rx).await; let reply = parse_process_started(&reply_msg).unwrap(); assert_eq!(reply.status, STATUS_CONFLICT); @@ -3577,7 +3624,7 @@ mod tests { let permits = u32::try_from(server.0.spawn_slots.available_permits()).unwrap(); let held = server.0.spawn_slots.acquire_many(permits).await.unwrap(); let endpoint = Arc::downgrade(&manager.endpoint); - manager.spawn(&spawn_message(10, 0, vec![b"sleep", b"30"]), None); + manager.spawn(&spawn_message(10, 0, vec![b"sleep", b"30"]), None, None); tokio::time::timeout(Duration::from_secs(1), manager.shutdown()) .await @@ -3654,7 +3701,7 @@ mod tests { #[tokio::test] async fn terminal_generation_is_held_until_exit_writer_guard_drops() { let (server, manager, mut rx) = manager(); - manager.spawn(&spawn_message(44, 0, vec![b"true"]), None); + manager.spawn(&spawn_message(44, 0, vec![b"true"]), None, None); let started = await_started(&mut rx).await; let (data, guard) = loop { let outbound = tokio::time::timeout(Duration::from_secs(5), rx.recv()) @@ -3693,6 +3740,7 @@ mod tests { vec![b"/bin/sh", b"-c", b"sleep .1; printf shared; sleep .1"], ), None, + None, ); let started = await_started(&mut first_rx).await; assert_eq!(started.status, STATUS_OK); @@ -3802,7 +3850,7 @@ mod tests { let owner = server.endpoint(owner_out); let (peer_out, mut peer_rx) = outbox(&server); let peer = server.endpoint(peer_out); - owner.spawn(&spawn_message(60, 0, vec![b"sleep", b"30"]), None); + owner.spawn(&spawn_message(60, 0, vec![b"sleep", b"30"]), None, None); let started = await_started(&mut owner_rx).await; let watch = msg_process_watch(ProcessWatch { @@ -3850,10 +3898,10 @@ mod tests { let first = server.endpoint(first_out); let (second_out, mut second_rx) = outbox(&server); let second = server.endpoint(second_out); - first.spawn(&spawn_message(62, 0, vec![b"sleep", b"30"]), None); + first.spawn(&spawn_message(62, 0, vec![b"sleep", b"30"]), None, None); assert_eq!(await_started(&mut first_rx).await.status, STATUS_OK); - second.spawn(&spawn_message(63, 0, vec![b"sleep", b"30"]), None); + second.spawn(&spawn_message(63, 0, vec![b"sleep", b"30"]), None, None); assert_eq!(await_started(&mut second_rx).await.status, STATUS_BUDGET); first.handle( @@ -3866,7 +3914,7 @@ mod tests { .unwrap(), ); assert_eq!(recv(&mut first_rx).await[0], S2C_PROCESS_CONTROLLED); - second.spawn(&spawn_message(63, 0, vec![b"sleep", b"30"]), None); + second.spawn(&spawn_message(63, 0, vec![b"sleep", b"30"]), None, None); assert_eq!(await_started(&mut second_rx).await.status, STATUS_OK); second.shutdown().await; @@ -3877,7 +3925,7 @@ mod tests { #[tokio::test] async fn unwatch_is_local_and_a_peer_watcher_can_control_the_child() { let (server, owner, mut owner_rx) = manager(); - owner.spawn(&spawn_message(17, 0, vec![b"sleep", b"30"]), None); + owner.spawn(&spawn_message(17, 0, vec![b"sleep", b"30"]), None, None); let started = await_started(&mut owner_rx).await; let (out, mut peer_rx) = outbox(&server); let peer = server.endpoint(out); @@ -3957,6 +4005,7 @@ mod tests { ], ), None, + None, ); let started = await_started(&mut owner_rx).await; let (max_frames, max_bytes) = server.outbox_limits(); @@ -4012,6 +4061,7 @@ mod tests { owner.spawn( &spawn_message(19, 0, vec![b"/bin/sh", b"-c", b"cat >/dev/null"]), None, + None, ); let started = await_started(&mut owner_rx).await; let (out, mut peer_rx) = outbox(&server); @@ -4122,7 +4172,7 @@ mod tests { #[tokio::test] async fn failed_watch_snapshot_does_not_publish_or_take_stdin() { let (server, owner, mut owner_rx) = manager(); - owner.spawn(&spawn_message(22, 0, vec![b"sleep", b"30"]), None); + owner.spawn(&spawn_message(22, 0, vec![b"sleep", b"30"]), None, None); let started = await_started(&mut owner_rx).await; let record = owner.get(22).unwrap(); owner.handle( @@ -4180,6 +4230,7 @@ mod tests { owner.spawn( &spawn_message(13, PROCESS_SPAWN_DETACHABLE, vec![b"true"]), None, + None, ); let started = await_started(&mut owner_rx).await; while recv(&mut owner_rx).await[0] != S2C_PROCESS_EXIT {} @@ -4213,6 +4264,7 @@ mod tests { manager.spawn( &spawn_message(46, PROCESS_SPAWN_DETACHABLE, vec![b"sleep", b"30"]), None, + None, ); let started = await_started(&mut rx).await; manager.handle( @@ -4236,7 +4288,7 @@ mod tests { #[tokio::test] async fn endpoint_shutdown_reaps_owned_process() { let (_server, manager, mut rx) = manager(); - manager.spawn(&spawn_message(43, 0, vec![b"sleep", b"30"]), None); + manager.spawn(&spawn_message(43, 0, vec![b"sleep", b"30"]), None, None); assert_eq!(await_started(&mut rx).await.status, STATUS_OK); tokio::time::timeout(Duration::from_secs(5), manager.shutdown()) .await @@ -4247,7 +4299,7 @@ mod tests { #[tokio::test] async fn owner_shutdown_publishes_owner_lost_exit_to_peer_watchers() { let (server, owner, mut owner_rx) = manager(); - owner.spawn(&spawn_message(48, 0, vec![b"sleep", b"30"]), None); + owner.spawn(&spawn_message(48, 0, vec![b"sleep", b"30"]), None, None); let started = await_started(&mut owner_rx).await; let (out, mut peer_rx) = outbox(&server); @@ -4311,6 +4363,7 @@ mod tests { ], ), None, + None, ); assert_eq!(await_started(&mut rx).await.status, STATUS_OK); let mut pid_bytes = Vec::new(); @@ -4369,6 +4422,7 @@ mod tests { manager.spawn( &spawn_message(61, 0, vec![b"/definitely/not/a/blit-test-program"]), None, + None, ); assert_eq!(await_started(&mut rx).await.status, STATUS_NOT_FOUND); assert!(manager.endpoint.state.lock().unwrap().slots.is_empty()); @@ -4380,7 +4434,7 @@ mod tests { ); } - manager.spawn(&spawn_message(61, 0, vec![b"true"]), None); + manager.spawn(&spawn_message(61, 0, vec![b"true"]), None, None); assert_eq!(await_started(&mut rx).await.status, STATUS_OK); while recv(&mut rx).await[0] != S2C_PROCESS_EXIT {} manager.shutdown().await; @@ -4397,6 +4451,7 @@ mod tests { vec![b"/bin/sh", b"-c", b"cat; printf err >&2"], ), None, + None, ); let started = await_started(&mut rx).await; assert_eq!(started.status, STATUS_OK); @@ -4473,6 +4528,7 @@ mod tests { vec![b"/bin/sh", b"-c", b"sleep .1; printf x; sleep 30"], ), None, + None, ); let started = await_started(&mut owner_rx).await; owner.handle( @@ -4556,6 +4612,7 @@ mod tests { manager.spawn( &spawn_message(65, PROCESS_SPAWN_DETACHABLE, vec![b"true"]), None, + None, ); let started = await_started(&mut rx).await; assert_eq!(started.status, STATUS_OK); @@ -4623,16 +4680,99 @@ mod tests { env: vec![], }; - let inherited = command_for(&request, None).output().await.unwrap(); + let inherited = command_for(&request, None, None).output().await.unwrap(); assert!(inherited.status.success()); assert_eq!(inherited.stdout, inherited_home.as_bytes()); request.env = vec![(b"HOME", b"/explicit")]; - let overridden = command_for(&request, None).output().await.unwrap(); + let overridden = command_for(&request, None, None).output().await.unwrap(); assert!(overridden.status.success()); assert_eq!(overridden.stdout, b"/explicit"); } + /// A GUI child has to reach *this* session's compositor, and the request's + /// own entries still have to win over the session's — the documented + /// contract is that explicit entries replace inherited ones. + #[tokio::test] + async fn session_env_reaches_the_child_and_yields_to_explicit_entries() { + let session = crate::app_env::session_env( + Some("/run/user/1000/wayland-7"), + None, + Some("unix:path=/run/user/1000/blit-bus"), + None, + None, + ); + let mut request = ProcessSpawnRequest { + nonce: 1, + process_id: 1, + flags: PROCESS_SPAWN_SESSION_ENV, + cwd_kind: PROCESS_CWD_DEFAULT, + src_pty_id: 0, + cwd: b"", + argv: vec![b"/bin/sh", b"-c", b"printf %s \"$WAYLAND_DISPLAY\""], + env: vec![], + }; + + let applied = command_for(&request, None, Some(&session)) + .output() + .await + .unwrap(); + assert!(applied.status.success()); + // The basename, never the path a client would fail to resolve. + assert_eq!(applied.stdout, b"wayland-7"); + + request.env = vec![(b"WAYLAND_DISPLAY", b"wayland-99")]; + let overridden = command_for(&request, None, Some(&session)) + .output() + .await + .unwrap(); + assert_eq!(overridden.stdout, b"wayland-99"); + } + + /// The host session's variables are as harmful as the session's own are + /// helpful: a child that inherits a `DISPLAY` blit cannot answer picks X11 + /// and maps no window at all. Nominating `DISPLAY` is `session_env`'s job; + /// actually stripping an *inherited* value is `command_for`'s, so prove that + /// half against a variable the test process is guaranteed to have. + #[tokio::test] + async fn a_removal_strips_a_variable_the_child_would_otherwise_inherit() { + let bridgeless = + crate::app_env::session_env(Some("/run/user/1000/wayland-7"), None, None, None, None); + assert!(bridgeless.remove.contains(&"DISPLAY")); + // With a bridge listening there is a display worth naming, so it must + // survive instead. + let bridged = crate::app_env::session_env( + Some("/run/user/1000/wayland-7"), + Some(":20"), + None, + None, + None, + ); + assert!(!bridged.remove.contains(&"DISPLAY")); + + std::env::var_os("HOME").expect("test process has HOME"); + let strips_home = crate::app_env::SessionEnv { + set: Vec::new(), + remove: vec!["HOME"], + }; + let request = ProcessSpawnRequest { + nonce: 1, + process_id: 1, + flags: PROCESS_SPAWN_SESSION_ENV, + cwd_kind: PROCESS_CWD_DEFAULT, + src_pty_id: 0, + cwd: b"", + argv: vec![b"/bin/sh", b"-c", b"printf %s \"${HOME-unset}\""], + env: vec![], + }; + let output = command_for(&request, None, Some(&strips_home)) + .output() + .await + .unwrap(); + assert!(output.status.success()); + assert_eq!(output.stdout, b"unset"); + } + #[tokio::test] async fn descriptors_opened_after_command_construction_are_not_inherited() { let request = ProcessSpawnRequest { @@ -4650,7 +4790,7 @@ mod tests { ], env: vec![], }; - let mut command = command_for(&request, None); + let mut command = command_for(&request, None, None); // F_DUPFD deliberately creates a descriptor without FD_CLOEXEC after // command_for captured its pre-exec closure. A parent-side descriptor @@ -4737,6 +4877,7 @@ mod windows_tests { manager.spawn( &spawn_message(1, argv.iter().map(Vec::as_slice).collect()), None, + None, ); let mut stdout = Vec::new(); @@ -4787,6 +4928,7 @@ mod windows_tests { manager.spawn( &spawn_message(2, argv.iter().map(Vec::as_slice).collect()), None, + None, ); loop { let message = recv(&mut rx).await; @@ -4831,6 +4973,7 @@ mod windows_tests { vec![(b"BLIT_PROCESS_TEST_DESCENDANT_PID", marker_text)], ), None, + None, ); loop { let message = recv(&mut rx).await; diff --git a/crates/server/src/pty/pty_unix.rs b/crates/server/src/pty/pty_unix.rs index 5954bcfe..f27f1f12 100644 --- a/crates/server/src/pty/pty_unix.rs +++ b/crates/server/src/pty/pty_unix.rs @@ -65,60 +65,20 @@ fn build_child_env( set(&mut env, "PATH", &next); } } - if let Some(wd) = wayland_display { - let wd_path = std::path::Path::new(wd); - if let Some(dir) = wd_path.parent() { - let xdg = std::env::var_os("XDG_RUNTIME_DIR"); - let needs_update = match &xdg { - Some(x) => std::path::Path::new(x) != dir, - None => true, - }; - if needs_update { - set(&mut env, "XDG_RUNTIME_DIR", &dir.to_string_lossy()); - } - } - // WAYLAND_DISPLAY must be just the socket filename (e.g. "wayland-2"), - // not a full path. Clients resolve it under XDG_RUNTIME_DIR. - let wd_name = wd_path - .file_name() - .map(|n| n.to_string_lossy()) - .unwrap_or_else(|| wd.into()); - set(&mut env, "WAYLAND_DISPLAY", &wd_name); - // The inherited DISPLAY was filtered out above: it belongs to the - // host's session, not this one. A toolkit left to choose for itself - // picks X11 and comes up with no window at all, so a GUI app typed - // at this shell gets the same steer the compositor gives the ones it - // spawns itself — and this session's own DISPLAY when an X11 bridge - // is running to answer it. - for (key, value) in crate::app_env::toolkit_env(x_display) { - set(&mut env, key, &value); - } - } - // The inherited address was filtered out above: both the server's audio - // bus and the host desktop bus belong to a different environment. The - // compositor-scoped bus activates portals on this Wayland display while - // still satisfying desktop apps (notably Spotify) that require a bus. - if let Some(address) = desktop_bus { - set(&mut env, "DBUS_SESSION_BUS_ADDRESS", address); - } - if let Some(ps) = pulse_server { - set(&mut env, "PULSE_SERVER", ps); - } else { - // No audio pipeline — point PULSE_SERVER at a path that will make - // libpulse fail immediately. Without this, libpulse falls back to - // autospawn (`pulseaudio --start`) which hangs in headless / - // container environments. Setting PULSE_SERVER explicitly also - // prevents inheriting a host PulseAudio server that would bypass - // blit's audio pipeline. - set(&mut env, "PULSE_SERVER", "/dev/null"); - } - // Set PIPEWIRE_REMOTE so native PipeWire clients (mpv, Firefox, etc.) - // can connect to our private PipeWire instance. WirePlumber is running - // as the session manager and handles linking streams to blit-sink. - // The path is absolute so it works regardless of the child's - // XDG_RUNTIME_DIR (which points at the Wayland socket directory). - if let Some(pr) = pipewire_remote { - set(&mut env, "PIPEWIRE_REMOTE", pr); + // The session half — compositor socket, toolkit steering, desktop bus, and + // audio sockets — is shared with native `PROCESS_SPAWN_SESSION_ENV` children + // so both routes reach the same display. Its `remove` list is already + // covered by the filter above, which drops the same host-session variables + // before they ever enter `env`. + let session = crate::app_env::session_env( + wayland_display, + x_display, + desktop_bus, + pulse_server, + pipewire_remote, + ); + for (key, value) in &session.set { + set(&mut env, key, value); } env.into_iter() .filter_map(|(k, v)| CString::new(format!("{k}={v}")).ok()) @@ -589,12 +549,12 @@ pub fn respond_to_queries( data: &[u8], size: (u16, u16), cursor: (u16, u16), -) -> Option { - let scan = crate::parse_terminal_queries(data, size, cursor); - for resp in scan.responses { +) -> crate::TerminalScan { + let mut scan = crate::parse_terminal_queries(data, size, cursor); + for resp in std::mem::take(&mut scan.responses) { pty_write_all(handle.master_fd, resp.as_bytes()); } - scan.osc7_cwd + scan } pub fn pty_reader(fd: PtyWriteTarget, tx: mpsc::Sender, notify: Arc) { @@ -850,6 +810,8 @@ pub fn spawn_pty( command: command.map(|s| s.to_owned()), cwd: dir.map(|s| s.to_owned()), osc7_cwd: None, + journal: crate::journal::CommandJournal::default(), + osc_carry: Vec::new(), }) } diff --git a/crates/server/src/pty/pty_windows.rs b/crates/server/src/pty/pty_windows.rs index d2861596..23d8d550 100644 --- a/crates/server/src/pty/pty_windows.rs +++ b/crates/server/src/pty/pty_windows.rs @@ -241,12 +241,12 @@ pub fn respond_to_queries( data: &[u8], size: (u16, u16), cursor: (u16, u16), -) -> Option { - let scan = crate::parse_terminal_queries(data, size, cursor); - for resp in scan.responses { +) -> crate::TerminalScan { + let mut scan = crate::parse_terminal_queries(data, size, cursor); + for resp in std::mem::take(&mut scan.responses) { pty_write_all(PtyWriteTarget(handle.input), resp.as_bytes()); } - scan.osc7_cwd + scan } pub(crate) struct SendHandle(pub(crate) HANDLE); @@ -511,6 +511,8 @@ pub fn spawn_pty( command: command.map(|s| s.to_owned()), cwd: dir.map(|s| s.to_owned()), osc7_cwd: None, + journal: crate::journal::CommandJournal::default(), + osc_carry: Vec::new(), }) } diff --git a/docs/design/env.md b/docs/design/env.md new file mode 100644 index 00000000..94c59df8 --- /dev/null +++ b/docs/design/env.md @@ -0,0 +1,93 @@ +# RFC: Server Environment + +- **Status:** Implemented (`FEATURE_ENV`, protocol feature bit 24) +- **Date:** 2026-08-18 +- **Companion to:** [processes.md](processes.md), [extensions.md](extensions.md), + [kv.md](kv.md), [../protocol.md](../protocol.md) + +## Summary + +One request, one reply: a client asks for the blit server's environment and +receives every variable, sorted by key. + +It exists because **a client has no other way to learn anything about the +session it is attached to.** Before this, `WAYLAND_DISPLAY` appeared exactly once +in the whole server — inside a dead function — and no message carried the +compositor socket, `XDG_RUNTIME_DIR`, the desktop bus address, or the audio +sockets. That knowledge belonged solely to the PTY spawn path. + +The immediate consumer is a Wasm extension. An extension's host ABI is five +imports — `send`, `recv`, `wait`, `clock`, `random` — with no filesystem, no +process, and no environment access; everything else it does, it does by speaking +this protocol as an ordinary client. So an extension that wants to enumerate +installed applications cannot read `XDG_DATA_DIRS` to find them. It can already +_read_ `/usr/share/applications` through the fs family, which accepts an +arbitrary root. It just could not find out where to look. + +Putting this in the protocol rather than adding a sixth wasm import is +deliberate: the ABI is kept minimal on purpose, and a protocol family gets +feature negotiation, a dispatch-level kill switch, and reach for every client +rather than extensions alone. + +## Wire + +Both directions use opcode `0x75`; the protocol is direction-local. + + C2S_ENV_GET [0x75][nonce:2] + S2C_ENV [0x75][nonce:2][status:1][count:2] + then count × [key_len:2][key:N][value_len:4][value:N] + +Records are ascending by key, so an unchanged environment encodes to identical +bytes. Keys and values are **raw bytes, not UTF-8**: a Unix environment carries +no such guarantee, and dropping an entry that failed to decode would be a worse +answer than handing it over as it is. + +A NUL in either half is `INVALID` — it cannot survive `execve`, so the codec +refuses to claim it round-tripped. A duplicate key is `INVALID` rather than +merged, since either resolution silently discards a value. Limits: key ≤ 4 KiB, +value ≤ 1 MiB, 8192 variables, 4 MiB of key and value bytes combined; exceeding +any of them answers `TOO_LARGE` with no entries. + +Every outcome is still one reply under the caller's nonce. A client waiting on +`S2C_ENV` is never left hanging, whatever went wrong. + +## Security + +**This hands the caller every credential the server was started with.** + +That is the whole posture, stated plainly. If the server's environment holds +`BLIT_PASSPHRASE` — which it does when `.env.local` is sourced — or a +`GITHUB_TOKEN`, an `ANTHROPIC_API_KEY`, or any other secret, then any client that +can reach this family reads it. There is no allowlist and no redaction. + +Two things bound it, neither of which should be mistaken for a sandbox: + +- The ceiling is the one the protocol already has. A client that can call + `ENV_GET` can also open a PTY, and a shell prints the same environment. This + family does not widen who can read what; it removes the need to spawn a process + to do it. +- **`BLIT_ENV=0` refuses the family at dispatch** with `PERMISSION` and no + entries. The feature bit stays advertised, following [kv.md](kv.md)'s + precedent, so a refusal is legible as an operator's decision rather than as an + old server, and a client can report the difference. + +The asymmetry worth understanding is _who_ is reading. A PTY child prints the +environment because a person typed a command; an extension reads it unattended, +at session start, from code the operator installed once. Persistent extensions +already require `--allow-persistent-extensions`, so the operator has opted in to +running that code — but they opted in to _running_ it, not necessarily to handing +it their credentials. An operator who wants the session-shaped values without the +secrets should set `BLIT_ENV=0` and rely on +[processes.md](processes.md)'s `PROCESS_SPAWN_SESSION_ENV`, which applies the +session environment to a child **server-side** without ever naming it on the +wire. + +## Non-goals + +- **No writes.** Nothing sets a server variable. The server's environment is + fixed at exec, and a family that mutated it would race every reader and every + in-flight spawn. +- **No watch.** There is no subscription: the value cannot change under a + running server, so a snapshot is complete by construction. +- **No per-client view.** Every caller sees the same environment. Scoping would + imply an identity model the protocol does not have. diff --git a/docs/design/extensions.md b/docs/design/extensions.md index 2e1745d9..0a9d0056 100644 --- a/docs/design/extensions.md +++ b/docs/design/extensions.md @@ -10,7 +10,7 @@ Blit should execute Rust extensions compiled to WebAssembly inside the server: ```bash -blit ext run --on prod extension.wasm arg1 arg2 +blit ext run --on prod builder extension.wasm arg1 arg2 ``` The client addresses the module by its full BLAKE3 digest. The server admits it @@ -170,9 +170,9 @@ in KV; PTY handles do not. ### `testgrid`: many isolated instances from one hash ```bash -blit ext run --on ci --restart always --persist --name test-unit testgrid.wasm unit -blit ext run --on ci --restart always --persist --name test-integration testgrid.wasm integration -blit ext run --on ci --restart always --persist --name test-web testgrid.wasm web +blit ext run --on ci --restart always --persist test-unit testgrid.wasm unit +blit ext run --on ci --restart always --persist test-integration testgrid.wasm integration +blit ext run --on ci --restart always --persist test-web testgrid.wasm web ``` The module uploads once, but each extension gets its own ID, arguments, thread, @@ -529,7 +529,7 @@ transient blocked extensions a recovery path. `PERSIST` stores the extension definition with enabled and desired-running both set. It implies `DETACH` and requires a unique durable name. Persistence does not itself alter the restart policy: the common cross-server daemon form is -`--restart always --persist --name NAME`. If the server shuts down while a +`--restart always --persist NAME`. If the server shuts down while a persistent extension is enabled and desired-running, the shutdown ends its current attempt without incrementing failure counters and a fresh attempt is launched after the next server has initialized its registries. @@ -1642,8 +1642,8 @@ setting. Operators create replicas as separate extensions, for example `worker-1`, `worker-2`, and `worker-3`, and manage them independently. ```bash -blit ext run --on prod --restart always --persist --name worker-1 worker.wasm queue-a -blit ext run --on prod --restart always --persist --name worker-2 worker.wasm queue-b +blit ext run --on prod --restart always --persist worker-1 worker.wasm queue-a +blit ext run --on prod --restart always --persist worker-2 worker.wasm queue-b blit ext list --on prod blit ext restart --on prod worker-1 ``` @@ -1766,6 +1766,8 @@ Client-to-server kinds: | 3 | `DATA` | `[payload:N]` | | 4 | `ACK` | `[bytes:8]` cumulative consumed payload bytes | | 5 | `CLOSE` | `[reason:1]` | +| 6 | `WATCH` | `[flags:1][count:2]` then `count` × `[name_len:2][name:N]` | +| 7 | `UNWATCH` | empty | `LISTEN.flags` has no version-1 bits and must be zero. `CONNECT.flags` bit 0 is `EXPECT_LISTENER_TOKEN` and appends `[listener_token:16]` after metadata; bits 1 @@ -1795,6 +1797,7 @@ Server-to-client kinds: | 3 | `DATA` | `[payload:N]` | | 4 | `ACK` | `[bytes:8]` cumulative consumed payload bytes | | 5 | `CLOSED` | `[reason:1][detail:N]` | +| 6 | `NAMES` | `[flags:1][count:2]` then `count` × `[name_len:2][name:N]` | Channel close reasons are: @@ -1855,6 +1858,36 @@ extension endpoint cannot make that hard outbox reservation, it is cancelled as a slow consumer and no listener is published; a `CONNECT` can therefore never observe a listener whose initial success reply was unreserved. +### Watching which names are served (`WATCH`) + +Feature bit **26** (`FEATURE_CHANNEL_WATCH`) advertises name watches. A `WATCH` +declares 1 to 32 distinct names on an unused client-created channel ID and is +answered immediately with `NAMES`: the declared names which have a listener +right now, in the order they were declared. `NAMES` is repeated whenever that +answer changes, and an empty list is an answer rather than silence. A watch +therefore replaces the connect-and-close probe that could only say what was +true once. + +A watch names what it follows rather than asking for the registry, so its +traffic is bounded by its own request and cannot scale with churn it has no +interest in — every extension attempt mints and drops a command listener. +Republication is suppressed when a watch's answer is byte-identical, which is +what makes an unrelated name's arrival cost a watcher nothing. + +`WATCH.flags` and `NAMES.flags` have no version-1 bits and must be zero. An +empty name list, a repeated name, a count above 32, or an odd (server-created) +ID is `OPENED(status = INVALID)`; an ID which is already live is `CONFLICT`, and +exhausting the per-endpoint watch budget is `BUDGET`. A watch holds its ID until +`UNWATCH`, which draws no reply and frees the ID for ordinary channel use; a +`NAMES` already in flight for a released ID must be ignored. `CLOSE` does not +end a watch. Watches are endpoint state: they are released with the endpoint and +never survive a reconnect. + +A name is released when its listener closes it or when the owning endpoint goes +away, and an extension's endpoint closes after the control call that disabled or +removed it returns. A watcher therefore learns of an extension going away from +that teardown rather than from the control call, which is a beat later. + No-reply operations have explicit stale-handle behavior. `DATA`, `ACK`, or `CLOSE` naming an absent, already-final, or terminally draining channel ID is ignored; a repeated `CLOSE` is therefore idempotent. `DATA` or `ACK` naming a @@ -2414,15 +2447,15 @@ needed. ```bash blit ext run --on prod extension.wasm arg1 arg2 blit ext run --on prod --restart on-failure extension.wasm arg1 -blit ext run --on prod --restart always --persist --name builder extension.wasm arg1 +blit ext run --on prod --restart always --persist builder extension.wasm arg1 ``` The canonical command grammar is -`blit ext run [RUN_OPTIONS] FILE [ARGS...]`. Every token +`blit ext run [RUN_OPTIONS] NAME FILE [ARGS...]`. Every token after `FILE` is passed verbatim as an extension argument, including tokens beginning with `-`; no `--` separator is required. Extension-run options such as -`--detach`, `--restart`, `--persist`, `--name`, and connection options such as -`--on` must therefore appear before `FILE`. +`--detach`, `--restart`, `--persist`, and connection options such as `--on` +must therefore appear before `NAME` and `FILE`, which are both positional. The CLI: @@ -2442,7 +2475,7 @@ is platform-specific; Unix shells see only the low eight bits (`0` through rather than the CLI process status. `--restart` accepts `never` (the default), `on-failure`, or `always`. -`--persist` requires `--name`, implies `--detach`, and stores an enabled, +`--persist` implies `--detach` and stores an enabled, desired-running definition for future blit server processes. It receives `PERMISSION` unless the selected server was started with `--allow-persistent-extensions`; the CLI reports that diff --git a/docs/design/fs-read.md b/docs/design/fs-read.md new file mode 100644 index 00000000..c5fe8e79 --- /dev/null +++ b/docs/design/fs-read.md @@ -0,0 +1,117 @@ +# One-shot reads (`FS_READ`) + +Read a fixed set of files, once, without a sync session. + +## Why it exists + +The fs family was built for an editor watching a tree: `FS_SYNC` establishes a +session, updates stream, and `FS_FETCH` names a `sync_id`. Everything else it +grew — `FS_SEARCH`, `FS_INDEX`, `FS_GREP` — asks about a root with no session, +because _discovery_ has nothing to watch. Reading did not get the same treatment, +so anything wanting a handful of files had two choices: sync a directory it did +not want watched, or spawn `/bin/sh -c 'cat …'`. + +The session supervisor took the second one, twice — every `.desktop` file at +startup, and every icon a panel asks for — and the shell brought its own problems +with it: quoting rules for every path, base64 to get bytes back through a text +stream, and `wc -c` to enforce a size limit the protocol should own. `FS_READ` is +the missing read. + +It grants nothing new. A client with `FEATURE_FS` can already read any file the +server user can, by syncing its directory and fetching it; this is the same +authority in one message instead of three. + +## Wire + +``` +C2S_FS_READ [0x4D][nonce:2][flags:1][max_bytes:4][group_count:2] + then group_count × ( [path_count:2] + then path_count × [path_len:2][path:N] ) + +S2C_FS_READ [0x48][nonce:2][status:1][count:2][records:LZ4] + records = count × [status:1][path_len:2][path:N][size:4][data:size] +``` + +Paths come in groups, and a group is one question. Without `FS_READ_FIRST` the +groups are read straight through and the grouping does not matter; with it each +group is answered by its own first readable path. Paths total 1 to +`FS_READ_MAX_PATHS` (512) however they are grouped. `max_bytes` is the per-file +ceiling, zero meaning `FS_READ_DEFAULT_BYTES` (1 MiB). + +`status` is the common registry (`FS_DONE_*`): `INVALID` for unknown flags, +`BUDGET` when too many reads are already in flight for this connection, `OK` +otherwise — including when every individual path failed, because that is an +answer about those paths rather than a failure of the request. + +Each record carries its own `FS_FILE_*`, so one unreadable path does not spoil +the rest: + +| Status | Meaning | +| ------------ | ------------------------------------------------------------------------------- | +| `OK` | content follows | +| `NOT_FOUND` | no such path | +| `UNREADABLE` | permission denied, or not a regular file — a directory is `FS_INDEX`'s business | +| `TOO_LARGE` | exists, not read: over `max_bytes`, or over what was left of the reply budget | +| `OTHER` | anything else I/O reported | + +A reply carries at most `FS_READ_MAX_TOTAL_BYTES` (8 MiB) of content. Files past +that are `TOO_LARGE` rather than silently dropped, so a caller can re-ask for the +remainder in smaller batches. + +### `FS_READ_NO_CONTENT` + +With `flags` bit 1 set, records name the file without carrying it: a status and a +path, an empty body, one stat per candidate instead of a read. `max_bytes` still +applies, so "exists, but too big to be what I am looking for" is still answered as +such. + +This is for a caller that only needs to know _where_ something is because it will +hand the path to whoever actually wants the bytes. The session supervisor resolves +icons this way and answers the panel with a path; the panel reads it itself, which +is the difference between artwork crossing a Wasm interpreter and not. A 30 KB +icon had to be base64`d into a JSON string in there, and that cost more than +everything else the panel did. + +### `FS_READ_FIRST` + +With `flags` bit 0 set, each group is answered by the first path in it that can +be read; missing, unreadable and oversized paths are stepped over rather than +reported. There is exactly one record per group, in group order, so answers align +with questions by position: a group that matched nothing carries +`FS_FILE_NOT_FOUND` and an empty path. + +This is the search-path question — _the first of these that exists, in my order +of preference_ — which is otherwise a round trip per candidate. The icon lookup +is exactly this: rank every directory on the icon path once, then ask for +`dir/name.svg`, `dir/name.png`, … in that order and take the first hit. Groups +are what make a screenful of them one message: one group per name, and the reply +carries one record per name. + +## `FS_INDEX` flags + +The listing side grew two flags for the same callers, on the byte that was +reserved: + +- `FS_INDEX_DIRS_ONLY` (bit 0) lists directories instead of files — the shape of + a tree without its contents, which is what a search path is. An icon theme is + fifty directories holding fifty thousand files. +- `FS_INDEX_FOLLOW_LINKS` (bit 1) descends through symlinked directories. Off by + default, because a tree's links can point anywhere. It exists because some + trees are _made_ of links: on a Nix system every directory under + `/run/current-system/sw/share/icons` is one, and a walk that stops at them + reports a theme's name and nothing inside it. + +Independently of the flags, an entry that is a symlink is now classified by what +it points at rather than skipped — one `stat` per link. Without it a Nix system +`applications` directory indexes as empty, every `.desktop` in it being a link +into the store. + +## Not in v1 + +- **No byte ranges.** Every caller so far wants whole files, and a range needs an + offset, a length, and a rule for a file that changed under it. +- **No stat that reports a size.** `FS_READ_NO_CONTENT` answers _which_ path and + whether it is under a ceiling, which is what a caller choosing between + candidates needs; the exact size has no reader yet. +- **No directory reads.** That is `FS_INDEX`, and conflating them would make one + message answer two shapes. diff --git a/docs/design/git.md b/docs/design/git.md index 3aaadfb0..b0ef652d 100644 --- a/docs/design/git.md +++ b/docs/design/git.md @@ -142,6 +142,7 @@ All integers little-endian; the 16 MiB frame limit and | C2S | `0xB2` | `GIT_BLAME` | `[nonce:2][repo_id:2][flags:1][oid:32][start_line:4][line_count:4][path_len:2][path:N]` | | C2S | `0xB3` | `GIT_REFLOG` | `[nonce:2][repo_id:2][flags:1][limit:2][after_pos:8][ref_len:2][ref:N]` | | C2S | `0xB4` | `GIT_FETCH` | `[nonce:2][repo_id:2][flags:1][timeout_ms:4][remote_len:2][remote:N][n_refspecs:2][(len:2, refspec:N)·N]` | +| C2S | `0xB5` | `GIT_WORKTREES` | `[nonce:2][repo_id:2][flags:1][after_pos:8]` | | S2C | `0xA0` | `GIT_REPO` | `[nonce:2][repo_id:2][status:1][oid_format:1][flags:1][workdir_len:2][workdir:N][gitdir_len:2][gitdir:N]` | | S2C | `0xA4` | `GIT_STATE` | `[repo_id:2][state_id:4][flags:1][records:LZ4]` | | S2C | `0xA5` | `GIT_CLOSED` | `[repo_id:2][reason:1]` | @@ -158,10 +159,14 @@ All integers little-endian; the 16 MiB frame limit and | S2C | `0xB2` | `GIT_BLAME` | `[nonce:2][status:1][flags:1][records:LZ4]` | | S2C | `0xB3` | `GIT_REFLOG` | `[nonce:2][status:1][flags:1][records:LZ4]` | | S2C | `0xB4` | `GIT_FETCH` | `[nonce:2][status:1][flags:1][records:LZ4]` | +| S2C | `0xB5` | `GIT_WORKTREES` | `[nonce:2][status:1][flags:1][records:LZ4]` | -One contiguous block, `0xA0`–`0xB4`, grouped by role — lifecycle, pushed +One contiguous block, `0xA0`–`0xB5`, grouped by role — lifecycle, pushed state, revision and log, object reads, then the repository-wide -operations — with `0xB5`–`0xBF` reserved for what comes next. The family +operations — with `0xB6`–`0xBF` reserved for what comes next. Contiguous is +load-bearing: the server admits the whole family by range, so an opcode +added outside it is dropped before any handler runs and shows up as a +request that never gets a reply. The family was renumbered out of `0x50` when the second pass broke the wire anyway: there was no reason to carry a split allocation forward, and the freed block is available to a future family. @@ -446,6 +451,14 @@ STATE_REMOTE 0x07: [kind:1][flags:1][name_len:2][name:N] this pull request belongs to" without parsing owner/name out of the worktree path and hoping the clone followed a convention. +WORKTREE_GEN 0x08: [kind:1][count:4][digest:8] + names the worktree *set* without describing it, so a client + knows when to refetch GIT_WORKTREES; count includes the main + worktree, digest is order-independent over the administrative + entries' names, gitdir mtimes and lock state. Emitted once per + snapshot, always — a 0/0 record is "no worktrees directory", + not "an old server that never said". See GIT_WORKTREES for + why the generation is pushed and the list is not. ``` **Pseudo-refs share the `STATE_REF` stream, and that is a migration @@ -1082,6 +1095,54 @@ parses, or transmits a secret; the fetch picks up whatever workaround already relied on. The difference is that the result comes back structured instead of scraped. +### `GIT_WORKTREES` + +```text +WORKTREE 0x01: [kind:1][flags:1][oid:32][path_len:2][path:N] + [branch_len:2][branch:N][lock_len:2][lock_reason:N] +``` + +Every worktree of the repository: the main one first, then the linked ones +in administrative-directory order. `flags` bit 0 `MAIN`, bit 1 `CURRENT`, +bit 2 `LOCKED`, bit 3 `PRUNABLE`, bit 4 `DETACHED`, bit 5 `BARE`. `branch` +is the full ref name HEAD is on, empty when `DETACHED`; `path` is the +escaped worktree root, empty when `BARE`; `lock_reason` is empty unless +`LOCKED`, and empty even then when the lock carried no reason. No request +flags are defined, and a non-zero `flags` is `INVALID` rather than ignored. + +The main worktree is resolved separately rather than taken from the +enumeration, because gix lists only _linked_ worktrees and does so relative +to whichever worktree the repo was opened through. Without that, a client +opened inside a linked worktree could not name the checkout it forked +from — which is the one it most wants to get back to, and the whole reason +this request exists. + +`CURRENT` is decided server-side from the repo handle's own worktree, so a +client does not have to compare paths it may have canonicalized differently +than the server did. A worktree whose checkout has been deleted behind +git's back is reported `PRUNABLE` rather than dropped: a row that cannot be +navigated to is precisely what a client needs to be told about. Its own +budget (`BLIT_GIT_WORKTREES_MAX`) rather than `entries_max`, because each +record costs opening that worktree's gitdir to resolve its HEAD — this +bounds repository opens, not bytes. Continuation is positional like +`GIT_REFLOG`'s: the enumeration has no name to resume from. + +**Staying live.** The list is a request, but a stale one would be worse +than none — a worktree removed an hour ago still offering to navigate +there. Adding, removing, moving or locking a worktree leaves every ref and +status record byte-identical, so per-subscriber identical-snapshot +suppression drops the push and a client refetching on ref moves is blind to +all of it (`git worktree remove` moves no ref at all). So each `GIT_STATE` +snapshot carries a `WORKTREE_GEN` record — a count plus an order-independent +digest over the administrative entries' names, their `gitdir` mtimes and +their lock state — and clients refetch when it moves. Only the generation is +pushed, never the list: the digest is a directory read and two stats per +worktree, affordable at every ref settle, while resolving HEADs is not. The +engine watches `/worktrees` recursively for it, armed separately +from the rest of the gitdir set because the directory only comes into +existence with the _first_ `git worktree add`, long after the one-shot +gitdir arm has run. + ## Mutation (proposed) The one part of this document that is a proposal rather than a contract. @@ -1127,6 +1188,7 @@ calls: | Open repos per connection | 16 | `BLIT_GIT_MAX_REPOS` | | Requests in flight per conn | 16 | `BLIT_GIT_MAX_INFLIGHT` | | Log subscriptions per repo | 64 | `BLIT_GIT_MAX_LOG_SUBS` | +| Worktrees per `GIT_WORKTREES` | 256 | `BLIT_GIT_WORKTREES_MAX` | | Ref settle window | 50 ms | `BLIT_GIT_REFS_LATENCY_MS` | | Status settle window | 500 ms | `BLIT_GIT_STATUS_LATENCY_MS` | | Blob / patch size cap | 16 MiB | `BLIT_GIT_BLOB_MAX` | diff --git a/docs/design/media-devices-portals.md b/docs/design/media-devices-portals.md index f1a53afe..7a7ca115 100644 --- a/docs/design/media-devices-portals.md +++ b/docs/design/media-devices-portals.md @@ -1201,6 +1201,9 @@ The full UI: title, app ID, size, and thumbnail; - shows a compact active-player row plus an expanded bounded list of all MPRIS players, with controls disabled exactly when their capabilities are absent; +- draws a progress bar for every player of known length, advancing it from the + extrapolated position on a clock of its own while the list is open, and + accepts a scrub as `SetPosition` only where CanSeek accompanies CanControl; - coordinates the selected writable player with browser Media Session and revokes every old action handler when selection changes; - keeps device labels local to the browser and never sends them to the server; diff --git a/docs/design/processes.md b/docs/design/processes.md index e790a38e..8b2656d1 100644 --- a/docs/design/processes.md +++ b/docs/design/processes.md @@ -153,16 +153,36 @@ they are compared with the native environment's case-insensitive key semantics after UTF-8 conversion, so spellings such as `Path` and `PATH` are duplicates. Duplicate keys are `INVALID`. -Spawn flags are bit 0 `MERGE_STDERR` and bit 1 `DETACHABLE`; any other bit is -`INVALID`. Process replies reuse the common status values, including -`PERMISSION`. Process-count or stream-window reservation failure returns -`PROCESS_STARTED(status = BUDGET)` before creating a child. +Spawn flags are bit 0 `MERGE_STDERR`, bit 1 `DETACHABLE`, and bit 2 +`SESSION_ENV`; any other bit is `INVALID`. Process replies reuse the common +status values, including `PERMISSION`. Process-count or stream-window +reservation failure returns `PROCESS_STARTED(status = BUDGET)` before creating a +child. + +`SESSION_ENV` is advertised separately from `FEATURE_PROCESS`, as +`FEATURE_PROCESS_SESSION_ENV`, because a server which does not understand the +flag answers with the same `INVALID` it uses for a corrupt frame — a client +cannot tell those apart by probing, so it must gate on the feature bit. The child inherits the complete server process environment. Explicit entries -add to it and replace inherited entries with the same key. There is no flag -which clears or filters that environment. Unix PTY creation additionally -rewrites terminal, compositor, desktop-bus, and audio variables; those -terminal-specific rewrites do not apply to native pipe children. +add to it and replace inherited entries with the same key. + +`SESSION_ENV` additionally applies the session environment: the compositor +socket (`WAYLAND_DISPLAY` as a basename, with `XDG_RUNTIME_DIR` naming its +directory), the toolkit steering every GUI application needs, the private +desktop bus, and this session's audio sockets. It is applied _before_ explicit +entries, so a request's own environment still wins. It also _removes_ the host +session's counterparts — `DISPLAY` when no X bridge is running to answer it, and +the bus and PipeWire addresses when the session has none — because a child that +inherits those reaches a different session, or picks X11 and maps no window at +all. Without the flag a GUI child gets whatever the server itself was started +with, which on a desktop host is the operator's own session rather than blit's. + +Requesting `SESSION_ENV` starts the compositor if it is not already running: a +supervised application can be the first thing in a session, and there would +otherwise be no socket to name. `PROCESS_LIST` never echoes the flag — the +catalog reports only `MERGE_STDERR` and `DETACHABLE`, because an older client +rejects an entire `PROCESS_LISTED` message carrying a flag bit it does not know. `PROCESS_LIST` returns one point-in-time catalog snapshot. It does not allocate an endpoint process slot or subscribe the caller. Entries are sorted by diff --git a/docs/design/term-journal.md b/docs/design/term-journal.md new file mode 100644 index 00000000..db16e57f --- /dev/null +++ b/docs/design/term-journal.md @@ -0,0 +1,269 @@ +# RFC: Per-command terminal journal + +- **Status:** Implemented (`FEATURE_TERM_JOURNAL`, protocol feature bit 28) +- **Date:** 2026-08-18 +- **Companion to:** [../protocol.md](../protocol.md), + [../shell-integration.md](../shell-integration.md) + +## Summary + +A long-lived shell PTY has no command boundary of its own. An agent that types +`make` into one today waits on a regex, sleeps, or dumps the whole scrollback, +and `terminal wait --pattern` used to match text that was already on screen +before the wait began. + +A shell that emits OSC 133 (FinalTerm semantic prompts) or OSC 633 (VS Code's +superset) already tells the terminal where each command starts, where its +output starts, and how it ended. This family turns that into a bounded ring of +records per PTY, addresses output by a monotonic sequence rather than a grid +row, and lets a client: + +- list the commands a terminal has run +- fetch one command's output +- read everything appended since a cursor +- block server-side until a command finishes + +```mermaid +flowchart LR + Shell["Shell"] -->|"OSC 133/633"| Pty["PTY output"] + Pty --> Driver["TerminalDriver"] + Pty --> Journal["CommandJournal"] + Driver -->|"seq_text"| Output["TERM_OUTPUT / TERM_SINCE"] + Journal --> List["TERM_JOURNAL"] + Journal --> Wait["TERM_JOURNAL_WAIT"] + Output --> Client + List --> Client + Wait --> Client +``` + +## Goals + +- Give an agent a command index and an output cursor that survive scrollback + eviction, rather than grid coordinates that move under it. +- Bound every reply with `max_bytes` and a truncation flag, so omitting a + limit cannot dump the default 10 000-line scrollback. +- Make `wait --pattern` match only output produced after the wait began. +- Cost nothing on a PTY whose shell emits no markers: no records, no extra + buffer, a scan that returns immediately on the first non-OSC byte. +- Stay backward compatible: older clients that do not negotiate the feature + bit are unaffected; existing opcode layouts do not change. + +## Non-goals + +- **No auto-injected shell integration.** Spawning a PTY with `ENV` / + `BASH_ENV` / `ZDOTDIR` clobbers user rc files and fights starship, oh-my-zsh, + and anything else that already emits OSC 133. The hooks live in + [shell-integration.md](../shell-integration.md) and are opt-in. A terminal + whose shell emits nothing keeps an empty journal; `blit terminal journal` + says so on stderr. +- **No `blit exec`.** Native non-PTY processes remain [processes.md](processes.md). + This family is for a shell the user (or an agent) is already typing into. +- **No apply of LSP edits, git writes, or batched fs mutations.** +- **No byte-range paging of a single grid row.** `seq_text` never splits a + row, so `max_bytes` is a soft cap that overshoots by at most one row. That + keeps paging monotonic. +- **No privilege boundary.** The journal is visible to every client that can + already `READ` the PTY. + +## Sequences + +A sequence is `rotated_lines + row`: the absolute index of a grid line since +the PTY was created. It names the same text for as long as that text is +retained. When scrollback evicts, `oldest_seq` rises and a read that started +below it comes back flagged `OUTPUT_EVICTED` with the surviving tail. + +`(seq, col)` together are a byte-exact cursor. `C2S_TERM_SINCE` with +`SINCE_PROBE` reports the current cursor and returns no text — that is how a +client starts following without first pulling everything already on screen. + +The alternate screen does not advance sequences. A read taken while the PTY +is on it comes back flagged `OUTPUT_ALT_SCREEN` and the cursor does not move; +full-screen programs are not command output. + +## Resize + +A sequence is `rotated_lines + row`. `row` is alacritty's `cursor.point.line` +(0 at the top of the viewport, negative in the history). A height change +moves lines between history and viewport, and alacritty updates `line` +accordingly: shrinking pushes viewport rows into history (`history_len` +grows, `cursor.line` falls); growing pulls them back (`grow_lines` does +`cursor.line += from_history`, `history_len` shrinks). + +`rotated_lines` follows the **signed** history-length delta so those two +moves cancel. Shrink increments it; grow decrements it. `saturating_sub` +would miss the grow direction, and every already-captured record plus the +live cursor would then name text `from_history` rows away. + +A column change rewraps, so a sequence no longer names the same bytes. Height +identity is the only correspondence resize preserves. + +## OSC 133 state machine + +Four markers, and a fifth that only OSC 633 speaks: + +| Marker | Meaning | +| ------- | ------------------------------------------------------------- | +| `A` | A prompt is being drawn. | +| `B` | The prompt is done; what follows is what the user types. | +| `C` | The command is about to run; what follows is its output. | +| `D` | The command finished. An optional `;status` is the exit code. | +| `633;E` | The command line, given verbatim (escapes: `\xHH`, `\\`). | + +`133;P`, `133;L`, and `633;P` are ignored and left for the emulator, which +drops them as it always has. A marker whose letter is followed by anything +other than end-of-payload or `;` is not ours (`133;Abc` is not `A`). + +The machine is total. Every transition either opens, closes, or ignores; there +is no input that leaves it stuck. + +| In | Event | Out | Record | +| --------------------- | ------- | ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Idle / Prompt / Input | `A` | Prompt | Running command, if any, closed `INCOMPLETE` (Ctrl-C, a shell that only emits `A`, a reset). | +| \* | `B` | Input, cursor saved | — | +| \* | `C` | Running | Previous running command closed `INCOMPLETE`. New record opened. Command line is `633;E` if seen, else the grid text between the saved `B` cursor and here. Empty command ⇒ `NO_COMMAND`. | +| Running | `D` | Idle | Record closed. `HAS_EXIT` only when `D` carried a status; a bare `D` is common and is not invented as zero. | +| Idle | `D` | Idle | Dropped: a status for a command that was never announced. | +| \* | `633;E` | unchanged | Held until the next `C`. | + +A PTY whose process exits while a command is running closes that command +`INCOMPLETE | PTY_EXITED`. A restarted PTY resets the journal: indices keep +climbing (`oldest_index = next_index`) so a client holding an old index sees +`NOT_FOUND` rather than the successor's output. + +Dialects latch. The first `A`/`B`/`C`/`D` a PTY sees chooses OSC 133 or OSC +633 for the rest of its life, so a shell that emits both is not counted twice. +`633;E` is additive and always taken — it only ever supplies text the other +dialect lacks. + +An unterminated OSC that straddles two PTY reads is held across the boundary, +capped at 512 bytes so a stream that opens an OSC and never closes it cannot +grow without bound. + +## Records + +Each record carries: + +- `index` — monotonic per PTY, never reused +- `flags` — `RUNNING`, `HAS_EXIT`, `NO_COMMAND`, `INCOMPLETE`, `EVICTED`, + `PTY_EXITED` +- `exit_code` — meaningful only under `HAS_EXIT` +- `[start_seq, end_seq)` — the output region. While the command runs, + `end_seq` is the live cursor and moves; once it completes it is frozen +- `started_ms` / `ended_ms` — Unix epoch milliseconds; `ended_ms` is 0 while + running +- `command` — recovered text, truncated to `BLIT_TERM_JOURNAL_CMD_MAX` + (default 4096) + +The ring holds at most `BLIT_TERM_JOURNAL_MAX` records (default 256). Eviction +drops the oldest and advances `oldest_index`. A record whose `start_seq` has +fallen below `oldest_seq` is flagged `EVICTED` on snapshot; fetching it +returns the surviving tail. + +## Wire protocol + +Feature bit **28** (`FEATURE_TERM_JOURNAL`). The family occupies the free +direction-local `0x50`–`0x53` block (filesystem ends at `0x4D` / `0x4B`; LSP +starts at `0x60`). + +`BLIT_TERM_JOURNAL=0` withholds the bit and refuses every nonce-bearing +request with `PERMISSION`, so a client that ignores feature bits still gets +its one reply. The PTY output path also skips the OSC scan. + +Every request is correlated by nonce. Status values are the common registry +(`OK`, `NOT_FOUND`, `PERMISSION`). `NOT_FOUND` is a missing PTY or a record +index below `oldest_index` or above `next_index`. + +### C2S + +``` +C2S_TERM_JOURNAL [0x50][nonce:2][pty_id:2][from_index:8][limit:2][flags:1] +C2S_TERM_OUTPUT [0x51][nonce:2][pty_id:2][index:8][max_bytes:4][flags:1] +C2S_TERM_SINCE [0x52][nonce:2][pty_id:2][from_seq:8][from_col:2][max_bytes:4][flags:1] +C2S_TERM_JOURNAL_WAIT [0x53][nonce:2][pty_id:2][index:8][timeout_ms:4] +``` + +`TERM_JOURNAL` flags bit 0 (`JOURNAL_TAIL`): `from_index` counts back from +the newest rather than up from the oldest, so `from_index = 0` means "the last +`limit`". + +`TERM_OUTPUT` / `TERM_JOURNAL_WAIT` accept `index = u64::MAX` +(`JOURNAL_INDEX_LATEST`) for the newest record. On an idle shell, a wait with +that sentinel latches onto the next command to start and never moves again. + +`TERM_SINCE` flags bit 0 (`SINCE_PROBE`): report the current cursor, return +no text. + +`max_bytes` is clamped server-side to `BLIT_TERM_OUTPUT_MAX` (default 1 MiB, +floor 4 KiB, ceiling 8 MiB). `timeout_ms` is clamped to 24 h. At most 32 +waits per connection; further ones are refused. + +### S2C + +``` +S2C_TERM_JOURNAL [0x50][nonce:2][pty_id:2][status:1][oldest_index:8][next_index:8][count:2][records…] +S2C_TERM_OUTPUT [0x51][nonce:2][pty_id:2][status:1][flags:1][start_seq:8][start_col:2][next_seq:8][next_col:2][text_len:4][text:N] +S2C_TERM_COMMAND [0x52][nonce:2][pty_id:2][status:1][record] +``` + +`S2C_TERM_OUTPUT` answers both `TERM_OUTPUT` and `TERM_SINCE`. `start_*` is +where the returned text actually begins after clamping; `next_*` is the cursor +to send back. Flags: `TRUNCATED`, `EVICTED`, `ALT_SCREEN`. + +`S2C_TERM_COMMAND` answers `TERM_JOURNAL_WAIT`. On timeout it carries the +record as it stands, still flagged `RUNNING`. A wait for a command that never +starts, or whose PTY disappears, comes back `NOT_FOUND`. + +A record on the wire: + +``` +[index:8][flags:1][exit_code:4][start_seq:8][end_seq:8][started_ms:8][ended_ms:8][cmd_len:2][command:N] +``` + +## CLI + +``` +blit terminal journal ID [--from INDEX] [--limit N] [--json] +blit terminal output ID [INDEX] [--wait SECONDS] [--max-bytes N] [--json] +blit terminal history ID --since CURSOR [--max-bytes N] [--json] +``` + +`journal` prints the newest 20 records by default. `output` defaults to the +newest command; `--wait` blocks server-side and exits with that command's +status (124 on timeout). `history --since` takes `SEQ`, `SEQ:COL`, `now`, or +`start`; the reply prints the next cursor so it can be fed back in. Default +`--max-bytes` is 256 KiB. + +`wait --pattern` probes the cursor first and then matches only `TERM_SINCE` +text. Against a server without the feature bit it falls back to scanning the +grid, which can still match pre-existing text — that path exists only so an +old remote keeps working. + +## `wait --pattern` + +The documented contract was "lines produced after the wait began". The +implementation re-scanned the whole grid on every `UPDATE`, so a pattern that +was already on screen returned immediately. + +The cursor is taken _before_ the subscription. Output that races between the +probe and the subscribe is recovered with one immediate `TERM_SINCE`; after +that, `UPDATE` is only the signal that there is something to read. Matching +runs against a pending buffer of new text, including a partial last line, so +a prompt that never ends in a newline (`Continue? [y/N]`) can still satisfy +the wait. + +## Security + +Reaching the socket is still equivalent to an interactive login as the server +user; this family grants no new authority. Command lines are whatever the +shell wrote to the PTY, including secrets typed at a prompt — the same +exposure `READ` already has. `BLIT_TERM_JOURNAL=0` is an ops kill switch, not +a privilege boundary. + +## Future work + +- Auto-inject the hooks for a PTY whose `$SHELL` is bash/zsh/fish, behind an + env flag, once the opt-in snippets have been lived with. +- OSC 133 `P` (prompt shown) / `L` (continuation) if a consumer appears. +- Push journal events (`S2C_TERM_COMMAND` unsolicited) so a client does not + have to wait or poll for a new index. Waiting is already server-side; this + would only save the listing round trip. diff --git a/docs/protocol.md b/docs/protocol.md index 615b5cef..acfa55b7 100644 --- a/docs/protocol.md +++ b/docs/protocol.md @@ -32,81 +32,86 @@ every pending operation as a connection error in that case. ## Client → Server (C2S) -| Opcode | Name | Layout | -| ------ | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `0x00` | `INPUT` | `[pty_id:2][data:N]` | -| `0x01` | `RESIZE` | `[pty_id:2][rows:2][cols:2]…` (batch, repeating triplets) | -| `0x02` | `SCROLL` | `[pty_id:2][offset:4]` | -| `0x03` | `ACK` | (no payload) | -| `0x04` | `DISPLAY_RATE` | `[fps:2]` | -| `0x05` | `CLIENT_METRICS` | `[backlog:2][ack_ahead:2][apply_ms_x10:2]` | -| `0x06` | `MOUSE` | `[pty_id:2][type:1][button:1][col:2][row:2]` | -| `0x07` | `RESTART` | `[pty_id:2]` | -| `0x08` | `PING` | _(empty)_ — application-level keepalive | -| `0x09` | `CLIENT_LIST` | `[nonce:2]` — enumerate connected clients | -| `0x0A` | `KICK` | `[nonce:2][client_id:8][reason:N]` — disconnect another client with a UTF-8 reason | -| `0x0B` | `CLIENT_WATCH` | `[nonce:2]` — subscribe to live client-catalog snapshots | -| `0x0C` | `CLIENT_UNWATCH` | `[nonce:2]` — stop the client-catalog subscription using this nonce | -| `0x0F` | `QUIT` | _(empty)_ — request server shutdown | -| `0x10` | `CREATE` | `[rows:2][cols:2][tag_len:2][tag:N]` | -| `0x11` | `FOCUS` | `[pty_id:2]` | -| `0x12` | `CLOSE` | `[pty_id:2]` | -| `0x13` | `SUBSCRIBE` | `[pty_id:2]` | -| `0x14` | `UNSUBSCRIBE` | `[pty_id:2]` | -| `0x15` | `SEARCH` | `[request_id:2][query:N]` | -| `0x16` | `CREATE_AT` | `[rows:2][cols:2][src_pty_id:2][tag_len:2][tag:N]` | -| `0x17` | `CREATE_N` | `[nonce:2][rows:2][cols:2][tag_len:2][tag:N]` | -| `0x18` | `CREATE2` | `[nonce:2][rows:2][cols:2][features:1][tag_len:2][tag:N][optional…]` | -| `0x19` | `READ` | `[nonce:2][pty_id:2][offset:4][limit:4][flags:1]` | -| `0x1A` | `KILL` | `[pty_id:2][signal:4][flags:1]` — send signal to a PTY's process group; `flags` optional | -| `0x1B` | `COPY_RANGE` | `[nonce:2][pty_id:2][start_tail:4][start_col:2][end_tail:4][end_col:2][flags:1]` | -| `0x1C` | `TERM_CWD` | `[nonce:2][pty_id:2]` — request a PTY's live working directory (see [Working directory tracking](#working-directory-tracking)) | -| `0x1D` | `DEADLINE` | `[pty_id:2][ms:4]` — arm or refresh a server-enforced deadline; `ms = 0` clears it | -| `0x1E` | `SCROLL_BY` | `[pty_id:2][delta:4 i32]` — move a scrolled view relative to where the server holds it (see [Scrollback](#scrollback)) | -| `0x20` | `SURFACE_INPUT` | `[surface_id:2][keycode:4][pressed:1][time_ms:4]` — `time_ms` is the browser `KeyboardEvent.timeStamp`, `0` when the sender synthesised the key | -| `0x21` | `SURFACE_POINTER` | `[surface_id:2][type:1][button:1][x:2][y:2]` | -| `0x22` | `SURFACE_POINTER_AXIS` | `[surface_id:2][axis:1][value:4]` — legacy scroll, superseded by `0x32` | -| `0x23` | `SURFACE_RESIZE` | `[surface_id:2][width:2][height:2][scale_120:2]` | -| `0x24` | `SURFACE_FOCUS` | `[surface_id:2]` | -| `0x25` | `CLIPBOARD_SET` | `[mime_len:2][mime:N][data_len:4][data:M]` | -| `0x26` | `SURFACE_LIST` | _(empty)_ — request list of compositor surfaces | -| `0x27` | `SURFACE_CAPTURE` | `[surface_id:2][format:1][quality:1]` — screenshot (0=PNG, 1=AVIF) | -| `0x28` | `SURFACE_SUBSCRIBE` | `[surface_id:2][codec:1][bandwidth:1][speed:1][width:2][height:2][max_fps:2]` | -| `0x29` | `SURFACE_UNSUBSCRIBE` | `[surface_id:2]` | -| `0x2A` | `SURFACE_ACK` | `[surface_id:2]` — acknowledge receipt of video frame | -| `0x2B` | `SURFACE_CLOSE` | `[surface_id:2]` — request close of Wayland surface | -| `0x2C` | `CLIPBOARD_LIST` | (no payload) | -| `0x2D` | `CLIENT_FEATURES` | `[codec_support:1]` — client capability advertisement | -| `0x2E` | `CLIPBOARD_GET` | `[mime_len:2][mime:N]` | -| `0x2F` | `SURFACE_TEXT` | `[surface_id:2][text:N]` — composed text input (UTF-8) | -| `0x30` | `AUDIO_SUBSCRIBE` | `[bitrate_kbps:2]` | -| `0x31` | `AUDIO_UNSUBSCRIBE` | (no payload) | -| `0x32` | `SURFACE_POINTER_AXIS2` | `[surface_id:2][flags:1][dx_x100:4][dy_x100:4][v120_x:2][v120_y:2]` — see [Scroll](#scroll) | -| `0x33` | `PRIMARY_SET` | `[mime_len:2][mime:N][data_len:4][data:M]` — take PRIMARY, see [Primary selection](#primary-selection) | -| `0x34` | `SURFACE_PREEDIT` | `[surface_id:2][cursor:2][text:N]` — composition in progress (UTF-8); `cursor` is a byte offset, empty text withdraws it | -| `0x35` | `SURFACE_DRAG_ENTER` | `[surface_id:2][x:2][y:2][mime_count:2][mime entries][optional item trailer]` — begin/retarget a drag; each entry is `[mime_len:2][mime:N]`; the append-only trailer is `[item_count:2][item MIME entries]`, see [Drag and drop](#drag-and-drop) | -| `0x36` | `SURFACE_DRAG_MOTION` | `[surface_id:2][x:2][y:2]` — move the drag | -| `0x37` | `SURFACE_DRAG_LEAVE` | `[surface_id:2]` — the drag left the surface | -| `0x38` | `SURFACE_DRAG_DROP` | `[surface_id:2][x:2][y:2][item_count:2][items]` — complete the drop; item is `[mime_len:2][mime][name_len:2][name][data_len:4][data]`, see [Drag and drop](#drag-and-drop) | -| `0x39` | `SURFACE_DRAG_CANCEL` | _(empty)_ — abort the drag (Escape / drag left the window) | -| `0x3A` | `SURFACE_TOUCH` | `[surface_id:2][phase:1][contact_count:1][time_ms:4][contacts…]`; contact is `[identifier:4 i32][x_x100:4 i32][y_x100:4 i32]`; `time_ms` is the browser `TouchEvent.timeStamp`, see [Direct touch](#direct-touch) | -| `0x3B` | `DESKTOP_SUBSCRIBE` | `[flags:1]`; bit 0 tray, bit 1 notifications, `0` unsubscribes; see [tray/notification design](design/tray-notifications.md#wire-protocol) | -| `0x3C` | `TRAY_EVENT` | `[tray_id:4][kind:1][menu_revision:4][value:4 i32][flags:1]`; see [tray/notification design](design/tray-notifications.md#client-to-server) | -| `0x3D` | `NOTIFICATION_EVENT` | `[notification_id:4][revision:4][kind:1][key_len:2][key:N]`; see [tray/notification design](design/tray-notifications.md#client-to-server) | -| `0x3E` | `MEDIA_CONTROL` | `[subtype:1][payload:N]`; viewer media leases, portal replies/stops, and MPRIS subscriptions/actions, see [media devices and portals](design/media-devices-portals.md#client-to-server-control) | -| `0x3F` | `MEDIA_DATA` | `[lease_id:4][sequence:4][capture_us:8][kind:1][codec:1][flags:1][fragment_index:2][fragment_count:2][frame_len:4][data:N]`, see [media devices and portals](design/media-devices-portals.md#media-data-and-fragmentation) | -| `0x40` | `FS_SYNC` | `[nonce:2][flags:2][latency_ms:2][inline_max:4][path_len:2][path:N]` + `[exclude_len:2][exclude:M]` if `EXCLUDE` + `[src_pty_id:2]` if `FROM_PTY`; `STAGING` roots the sync at the drag staging dir, see [Drag and drop](#drag-and-drop) | -| `0x41` | `FS_STOP` | `[sync_id:2]` | -| `0x42` | `FS_ACK` | `[sync_id:2][update_id:4]` — cumulative | -| `0x43` | `FS_FETCH` | `[nonce:2][sync_id:2][path_len:2][path:N]` | -| `0x44` | `FS_WRITE` | `[nonce:2][sync_id:2][flags:1][base:16][mode:4][content_kind:1][path_len:2][path:N][content:LZ4]` — CAS content upsert ([design/fs-write.md](design/fs-write.md)) | -| `0x45` | `FS_OP` | `[nonce:2][sync_id:2][op:1][flags:1][base:16][mode:4][a_len:2][a:N][b_len:2][b:N]` — mkdir/remove/rename/symlink/hardlink ([design/fs-write.md](design/fs-write.md)) | -| `0x46` | `FS_SEARCH` | `[nonce:2][limit:2][root_len:2][root:N][query_len:2][query:M]` — server-side fuzzy file search ([design/fs-search.md](design/fs-search.md)) | -| `0x47` | `FS_INDEX` | `[nonce:2][flags:1][root_len:2][root:N]` — candidate list for client-side search ([design/fs-search.md](design/fs-search.md)) | -| `0x49` | `FS_UPLOAD_BEGIN` | `[nonce:2][sync_id:2][flags:1][base:16][mode:4][size:8][path_len:2][path:N]` — begin a chunked upload; `base` is the `FS_WRITE` CAS precondition | -| `0x4A` | `FS_UPLOAD_CHUNK` | `[upload_id:2][offset:8][data:LZ4]` — sequential append; `offset` must equal the bytes accepted so far | -| `0x4B` | `FS_UPLOAD_FINISH` | `[nonce:2][upload_id:2]` — land the upload (terminates it either way) | -| `0x4C` | `FS_UPLOAD_CANCEL` | `[upload_id:2]` — abort the upload; no reply | +| Opcode | Name | Layout | +| ------ | ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `0x00` | `INPUT` | `[pty_id:2][data:N]` | +| `0x01` | `RESIZE` | `[pty_id:2][rows:2][cols:2]…` (batch, repeating triplets) | +| `0x02` | `SCROLL` | `[pty_id:2][offset:4]` | +| `0x03` | `ACK` | (no payload) | +| `0x04` | `DISPLAY_RATE` | `[fps:2]` | +| `0x05` | `CLIENT_METRICS` | `[backlog:2][ack_ahead:2][apply_ms_x10:2]` | +| `0x06` | `MOUSE` | `[pty_id:2][type:1][button:1][col:2][row:2]` | +| `0x07` | `RESTART` | `[pty_id:2]` | +| `0x08` | `PING` | _(empty)_ — application-level keepalive | +| `0x09` | `CLIENT_LIST` | `[nonce:2]` or `[nonce:2][flags:1]` — enumerate connected clients; `flags` bit 0 (`WANT_ORIGIN`) asks for `S2C_CLIENT_LIST2` | +| `0x0A` | `KICK` | `[nonce:2][client_id:8][reason:N]` — disconnect another client with a UTF-8 reason | +| `0x0B` | `CLIENT_WATCH` | `[nonce:2]` or `[nonce:2][flags:1]` — subscribe to live client-catalog snapshots, under the same flags as `CLIENT_LIST` | +| `0x0C` | `CLIENT_UNWATCH` | `[nonce:2]` — stop the client-catalog subscription using this nonce | +| `0x0F` | `QUIT` | _(empty)_ — request server shutdown | +| `0x10` | `CREATE` | `[rows:2][cols:2][tag_len:2][tag:N]` | +| `0x11` | `FOCUS` | `[pty_id:2]` | +| `0x12` | `CLOSE` | `[pty_id:2]` | +| `0x13` | `SUBSCRIBE` | `[pty_id:2]` | +| `0x14` | `UNSUBSCRIBE` | `[pty_id:2]` | +| `0x15` | `SEARCH` | `[request_id:2][query:N]` | +| `0x16` | `CREATE_AT` | `[rows:2][cols:2][src_pty_id:2][tag_len:2][tag:N]` | +| `0x17` | `CREATE_N` | `[nonce:2][rows:2][cols:2][tag_len:2][tag:N]` | +| `0x18` | `CREATE2` | `[nonce:2][rows:2][cols:2][features:1][tag_len:2][tag:N][optional…]` | +| `0x19` | `READ` | `[nonce:2][pty_id:2][offset:4][limit:4][flags:1]` | +| `0x1A` | `KILL` | `[pty_id:2][signal:4][flags:1]` — send signal to a PTY's process group; `flags` optional | +| `0x1B` | `COPY_RANGE` | `[nonce:2][pty_id:2][start_tail:4][start_col:2][end_tail:4][end_col:2][flags:1]` | +| `0x1C` | `TERM_CWD` | `[nonce:2][pty_id:2]` — request a PTY's live working directory (see [Working directory tracking](#working-directory-tracking)) | +| `0x1D` | `DEADLINE` | `[pty_id:2][ms:4]` — arm or refresh a server-enforced deadline; `ms = 0` clears it | +| `0x1E` | `SCROLL_BY` | `[pty_id:2][delta:4 i32]` — move a scrolled view relative to where the server holds it (see [Scrollback](#scrollback)) | +| `0x20` | `SURFACE_INPUT` | `[surface_id:2][keycode:4][pressed:1][time_ms:4]` — `time_ms` is the browser `KeyboardEvent.timeStamp`, `0` when the sender synthesised the key | +| `0x21` | `SURFACE_POINTER` | `[surface_id:2][type:1][button:1][x:2][y:2]` | +| `0x22` | `SURFACE_POINTER_AXIS` | `[surface_id:2][axis:1][value:4]` — legacy scroll, superseded by `0x32` | +| `0x23` | `SURFACE_RESIZE` | `[surface_id:2][width:2][height:2][scale_120:2]` | +| `0x24` | `SURFACE_FOCUS` | `[surface_id:2]` | +| `0x25` | `CLIPBOARD_SET` | `[mime_len:2][mime:N][data_len:4][data:M]` | +| `0x26` | `SURFACE_LIST` | _(empty)_ — request list of compositor surfaces | +| `0x27` | `SURFACE_CAPTURE` | `[surface_id:2][format:1][quality:1]` — screenshot (0=PNG, 1=AVIF) | +| `0x28` | `SURFACE_SUBSCRIBE` | `[surface_id:2][codec:1][bandwidth:1][speed:1][width:2][height:2][max_fps:2]` | +| `0x29` | `SURFACE_UNSUBSCRIBE` | `[surface_id:2]` | +| `0x2A` | `SURFACE_ACK` | `[surface_id:2]` — acknowledge receipt of video frame | +| `0x2B` | `SURFACE_CLOSE` | `[surface_id:2]` — request close of Wayland surface | +| `0x2C` | `CLIPBOARD_LIST` | (no payload) | +| `0x2D` | `CLIENT_FEATURES` | `[codec_support:1]` — client capability advertisement | +| `0x2E` | `CLIPBOARD_GET` | `[mime_len:2][mime:N]` | +| `0x2F` | `SURFACE_TEXT` | `[surface_id:2][text:N]` — composed text input (UTF-8) | +| `0x30` | `AUDIO_SUBSCRIBE` | `[bitrate_kbps:2]` | +| `0x31` | `AUDIO_UNSUBSCRIBE` | (no payload) | +| `0x32` | `SURFACE_POINTER_AXIS2` | `[surface_id:2][flags:1][dx_x100:4][dy_x100:4][v120_x:2][v120_y:2]` — see [Scroll](#scroll) | +| `0x33` | `PRIMARY_SET` | `[mime_len:2][mime:N][data_len:4][data:M]` — take PRIMARY, see [Primary selection](#primary-selection) | +| `0x34` | `SURFACE_PREEDIT` | `[surface_id:2][cursor:2][text:N]` — composition in progress (UTF-8); `cursor` is a byte offset, empty text withdraws it | +| `0x35` | `SURFACE_DRAG_ENTER` | `[surface_id:2][x:2][y:2][mime_count:2][mime entries][optional item trailer]` — begin/retarget a drag; each entry is `[mime_len:2][mime:N]`; the append-only trailer is `[item_count:2][item MIME entries]`, see [Drag and drop](#drag-and-drop) | +| `0x36` | `SURFACE_DRAG_MOTION` | `[surface_id:2][x:2][y:2]` — move the drag | +| `0x37` | `SURFACE_DRAG_LEAVE` | `[surface_id:2]` — the drag left the surface | +| `0x38` | `SURFACE_DRAG_DROP` | `[surface_id:2][x:2][y:2][item_count:2][items]` — complete the drop; item is `[mime_len:2][mime][name_len:2][name][data_len:4][data]`, see [Drag and drop](#drag-and-drop) | +| `0x39` | `SURFACE_DRAG_CANCEL` | _(empty)_ — abort the drag (Escape / drag left the window) | +| `0x3A` | `SURFACE_TOUCH` | `[surface_id:2][phase:1][contact_count:1][time_ms:4][contacts…]`; contact is `[identifier:4 i32][x_x100:4 i32][y_x100:4 i32]`; `time_ms` is the browser `TouchEvent.timeStamp`, see [Direct touch](#direct-touch) | +| `0x3B` | `DESKTOP_SUBSCRIBE` | `[flags:1]`; bit 0 tray, bit 1 notifications, `0` unsubscribes; see [tray/notification design](design/tray-notifications.md#wire-protocol) | +| `0x3C` | `TRAY_EVENT` | `[tray_id:4][kind:1][menu_revision:4][value:4 i32][flags:1]`; see [tray/notification design](design/tray-notifications.md#client-to-server) | +| `0x3D` | `NOTIFICATION_EVENT` | `[notification_id:4][revision:4][kind:1][key_len:2][key:N]`; see [tray/notification design](design/tray-notifications.md#client-to-server) | +| `0x3E` | `MEDIA_CONTROL` | `[subtype:1][payload:N]`; viewer media leases, portal replies/stops, and MPRIS subscriptions/actions, see [media devices and portals](design/media-devices-portals.md#client-to-server-control) | +| `0x3F` | `MEDIA_DATA` | `[lease_id:4][sequence:4][capture_us:8][kind:1][codec:1][flags:1][fragment_index:2][fragment_count:2][frame_len:4][data:N]`, see [media devices and portals](design/media-devices-portals.md#media-data-and-fragmentation) | +| `0x40` | `FS_SYNC` | `[nonce:2][flags:2][latency_ms:2][inline_max:4][path_len:2][path:N]` + `[exclude_len:2][exclude:M]` if `EXCLUDE` + `[src_pty_id:2]` if `FROM_PTY`; `STAGING` roots the sync at the drag staging dir, see [Drag and drop](#drag-and-drop) | +| `0x41` | `FS_STOP` | `[sync_id:2]` | +| `0x42` | `FS_ACK` | `[sync_id:2][update_id:4]` — cumulative | +| `0x43` | `FS_FETCH` | `[nonce:2][sync_id:2][path_len:2][path:N]` | +| `0x44` | `FS_WRITE` | `[nonce:2][sync_id:2][flags:1][base:16][mode:4][content_kind:1][path_len:2][path:N][content:LZ4]` — CAS content upsert ([design/fs-write.md](design/fs-write.md)) | +| `0x45` | `FS_OP` | `[nonce:2][sync_id:2][op:1][flags:1][base:16][mode:4][a_len:2][a:N][b_len:2][b:N]` — mkdir/remove/rename/symlink/hardlink ([design/fs-write.md](design/fs-write.md)) | +| `0x46` | `FS_SEARCH` | `[nonce:2][limit:2][root_len:2][root:N][query_len:2][query:M]` — server-side fuzzy file search ([design/fs-search.md](design/fs-search.md)) | +| `0x47` | `FS_INDEX` | `[nonce:2][flags:1][root_len:2][root:N]` — candidate list for client-side search; `flags` is `DIRS_ONLY`/`FOLLOW_LINKS` ([design/fs-search.md](design/fs-search.md), [design/fs-read.md](design/fs-read.md)) | +| `0x49` | `FS_UPLOAD_BEGIN` | `[nonce:2][sync_id:2][flags:1][base:16][mode:4][size:8][path_len:2][path:N]` — begin a chunked upload; `base` is the `FS_WRITE` CAS precondition | +| `0x4A` | `FS_UPLOAD_CHUNK` | `[upload_id:2][offset:8][data:LZ4]` — sequential append; `offset` must equal the bytes accepted so far | +| `0x4B` | `FS_UPLOAD_FINISH` | `[nonce:2][upload_id:2]` — land the upload (terminates it either way) | +| `0x4C` | `FS_UPLOAD_CANCEL` | `[upload_id:2]` — abort the upload; no reply | +| `0x4D` | `FS_READ` | `[nonce:2][flags:1][max_bytes:4][group_count:2] repeated{ [path_count:2] repeated{ [path_len:2][path:N] } }` — one-shot read, no sync; a group is one question, and `NO_CONTENT` names files instead of carrying them ([design/fs-read.md](design/fs-read.md)) | +| `0x50` | `TERM_JOURNAL` | `[nonce:2][pty_id:2][from_index:8][limit:2][flags:1]` — list OSC 133 command records; `flags` bit 0 (`JOURNAL_TAIL`) counts `from_index` back from the newest ([design/term-journal.md](design/term-journal.md)) | +| `0x51` | `TERM_OUTPUT` | `[nonce:2][pty_id:2][index:8][max_bytes:4][flags:1]` — one command's output; `index = u64::MAX` is the newest ([design/term-journal.md](design/term-journal.md)) | +| `0x52` | `TERM_SINCE` | `[nonce:2][pty_id:2][from_seq:8][from_col:2][max_bytes:4][flags:1]` — output since a sequence cursor; `flags` bit 0 (`SINCE_PROBE`) reports the cursor and returns no text ([design/term-journal.md](design/term-journal.md)) | +| `0x53` | `TERM_JOURNAL_WAIT` | `[nonce:2][pty_id:2][index:8][timeout_ms:4]` — block until that command finishes; `index = u64::MAX` waits on whatever is running, or the next one to start ([design/term-journal.md](design/term-journal.md)) | **Notes:** @@ -161,6 +166,30 @@ can produce an update that often even when topology is unchanged; between samples only a real topology change publishes. Multiple watch nonces per client are valid. +A `CLIENT_LIST` or `CLIENT_WATCH` may carry one trailing flags byte once +`FEATURE_CLIENT_ORIGIN` is advertised. Its bit 0 (`CLIENT_LIST_WANT_ORIGIN`) +asks for `S2C_CLIENT_LIST2`, which is the same message with +`[origin_kind:1][origin_len:2][origin:N]` appended to every entry. +`origin_kind` is `0` for an ordinary client — a browser, a CLI, a forwarder, +with an empty payload — or `1` for the connection belonging to a running +extension attempt, whose payload is +`[extension_id:8][definition_revision:8][attempt:8][task_id:4][name:N]`. The +name is the durable name of a persistent definition or the label a transient +`ext run` carried, empty when it has neither, and is captured when the +connection opens, so it names the definition the attempt started from. +`origin_len` is what a reader skips past a kind it does not know, which is how +a later kind can carry a payload without a third opcode. + +The wider entry needs its own opcode because both shipped parsers reject a +catalog with bytes left over — it has to be unreadable to them by construction. +For the same reason, a flags byte is only safe once the feature bit is seen: +every server answers a client-control request with unexpected trailing bytes +with `INVALID`, and so does this one for a flag it does not recognise. Any +unknown flag bit is refused rather than ignored, so a client asking for a shape +the server cannot produce hears about it instead of misreading the reply. +`CLIENT_UNWATCH` takes no flags: a watch keeps the shape it was opened with for +its whole life. + `S2C_KICK_RESULT` is the status reply for the whole client-control family, not only for `C2S_KICK`: a malformed `CLIENT_LIST` / `CLIENT_WATCH` / `CLIENT_UNWATCH` is answered with `INVALID` under the sender's nonce rather @@ -170,8 +199,9 @@ A request too short to carry a nonce (fewer than three bytes) is the one case the server drops, because the nonce would be a guess. The `blit client list` CLI filters its own short-lived connection from its -output, while persistent clients can use `self_id` to identify their own live -record. Client IDs are a per-process counter, not a capability or a stable +output, and prints an `ORIGIN` column (`network`, `ext:`, `ext:id:`) +whenever the server offers origins — `unknown` when it does not. Persistent +clients can use `self_id` to identify their own live record. Client IDs are a per-process counter, not a capability or a stable identity: they are only meaningful for the life of the server process, `HELLO.boot_generation` identifies that lifetime, and a client can infer from its own ID how many connections preceded it. @@ -283,6 +313,7 @@ shared sizing input without a `SURFACE_RESIZE` entry. | `0x12` | `CLIENT_LIST` | `[nonce:2][self_id:8][count:4][client:N]…` — sorted connection records, including the requester | | `0x13` | `KICK_RESULT` | `[nonce:2][status:1][detail:N]` — correlated result of `C2S_KICK` | | `0x14` | `KICKED` | `[reason:N]` — another client kicked this connection; the server closes it after delivery | +| `0x15` | `CLIENT_LIST2` | `CLIENT_LIST` with `[origin_kind:1][origin_len:2][origin:N]` on every record — answer to a request carrying `WANT_ORIGIN` | | `0x20` | `SURFACE_CREATED` | `[surface_id:2][parent_id:2][w:2][h:2][title_len:2][title:N][app_id_len:2][app_id:M]` | | `0x21` | `SURFACE_DESTROYED` | `[surface_id:2]` | | `0x22` | `SURFACE_FRAME` | `[surface_id:2][timestamp:4][flags:1][w:2][h:2][data:N]` | @@ -312,39 +343,53 @@ shared sizing input without a `SURFACE_RESIZE` entry. | `0x44` | `FS_DONE` | `[nonce:2][status:1][hash:16][mtime_ns:8]` — one per `FS_WRITE`/`FS_OP` ([design/fs-write.md](design/fs-write.md)) | | `0x45` | `FS_SEARCH` | `[nonce:2][status:1][count:2] repeated{ [path_len:2][path:N] }` ([design/fs-search.md](design/fs-search.md)) | | `0x46` | `FS_INDEX` | `[nonce:2][status:1][flags:1][count:4][paths:LZ4]` ([design/fs-search.md](design/fs-search.md)) | +| `0x48` | `FS_READ` | `[nonce:2][status:1][count:2][records:LZ4]` — a status and content per requested path ([design/fs-read.md](design/fs-read.md)) | | `0x49` | `FS_UPLOAD_BEGIN` | `[nonce:2][status:1][upload_id:2][hash:16][mtime_ns:8]` — `upload_id` meaningful only on `OK`; `hash` is the current on-disk hash on `CONFLICT` | | `0x4A` | `FS_UPLOAD_CHUNK` | `[upload_id:2][status:1][received:8]` — per-chunk ack/progress; `received` is the resume point on `OFFSET_MISMATCH` | | `0x4B` | `FS_UPLOAD_FINISH` | `[nonce:2][status:1][hash:16][mtime_ns:8]` — the `FS_DONE` payload on success (zeroes otherwise) | +| `0x50` | `TERM_JOURNAL` | `[nonce:2][pty_id:2][status:1][oldest_index:8][next_index:8][count:2][records…]` — answer to `C2S_TERM_JOURNAL` ([design/term-journal.md](design/term-journal.md)) | +| `0x51` | `TERM_OUTPUT` | `[nonce:2][pty_id:2][status:1][flags:1][start_seq:8][start_col:2][next_seq:8][next_col:2][text_len:4][text:N]` — answers `TERM_OUTPUT` and `TERM_SINCE`; `next_*` is the cursor to resume from ([design/term-journal.md](design/term-journal.md)) | +| `0x52` | `TERM_COMMAND` | `[nonce:2][pty_id:2][status:1][record]` — answer to `C2S_TERM_JOURNAL_WAIT`; on timeout the record is still flagged `RUNNING` ([design/term-journal.md](design/term-journal.md)) | **Notes:** `S2C_HELLO` is the first message sent on every new connection. `version` is the server's protocol version. `boot_generation` is an opaque little-endian identifier generated once per server process; clients can compare it across reconnects to detect a server restart. `server_version` is the server's release string (its crate version, e.g. `0.40.1`) — informational only: feature negotiation always goes through the feature bits, never a version comparison. Both trailing fields were appended without a protocol bump, so legacy servers omit them and clients must treat a short `HELLO` as valid. `features` is a 4-byte bitmask: -| Bit | Name | Meaning | -| --- | -------------------- | --------------------------------------------------------------- | -| 0 | `CREATE_NONCE` | Server supports `CREATE2` / `CREATED_N` with nonce correlation | -| 1 | `RESTART` | Server supports `C2S_RESTART` to respawn exited PTYs | -| 2 | `RESIZE_BATCH` | Server accepts batched resize entries in a single `C2S_RESIZE` | -| 3 | `COPY_RANGE` | Server supports range-based text copy | -| 4 | `COMPOSITOR` | Server supports headless Wayland compositor | -| 5 | `AUDIO` | Server supports audio forwarding (PipeWire capture + Opus) | -| 6 | `FS` | Server supports the `FS_*` filesystem sync family | -| 7 | `GIT` | Server supports the `GIT_*` git introspection family | -| 8 | `LSP` | Server supports the `LSP_*` language intelligence family | -| 9 | `KV` | Server supports the `KV_*` key-value family | -| 10 | `NET` | Server supports the `NET_*` network-relay family | -| 11 | `EXTENSION` | Proposed: Wasmi extension lifecycle, events, and commands | -| 12 | `CHANNEL` | Proposed: server supports bidirectional named channels | -| 13 | `PROCESS` | Server supports native non-PTY child processes | -| 14 | `CREATE_STATUS` | `CREATE2(WANT_STATUS)` receives an explicit failure | -| 15 | `KILL_MODE` | `KILL`/`CLOSE` reach the process group; `KILL` takes `flags` | -| 16 | `PTY_DEADLINE` | `C2S_DEADLINE`, `CREATE2(HAS_DEADLINE)`, and `EXITED.reason` | -| 17 | `SCROLL_BY` | Scrollback holds still: `S2C_SCROLL_OFFSET` and `C2S_SCROLL_BY` | -| 18 | `SURFACE_TOUCH` | Server accepts direct contacts and exposes `wl_seat.touch` | -| 19 | `SURFACE_TEXT_INPUT` | Server forwards committed `zwp_text_input_v3` state | -| 20 | `CLIENT_CONTROL` | Enumerate connections and kick another client with a reason | -| 21 | `DESKTOP` | Compositor tray/notification state bridge and core API are live | -| 22 | `DESKTOP_MEDIA` | Viewer media, portals, and MPRIS control family is understood | +| Bit | Name | Meaning | +| --- | --------------------- | --------------------------------------------------------------- | +| 0 | `CREATE_NONCE` | Server supports `CREATE2` / `CREATED_N` with nonce correlation | +| 1 | `RESTART` | Server supports `C2S_RESTART` to respawn exited PTYs | +| 2 | `RESIZE_BATCH` | Server accepts batched resize entries in a single `C2S_RESIZE` | +| 3 | `COPY_RANGE` | Server supports range-based text copy | +| 4 | `COMPOSITOR` | Server supports headless Wayland compositor | +| 5 | `AUDIO` | Server supports audio forwarding (PipeWire capture + Opus) | +| 6 | `FS` | Server supports the `FS_*` filesystem sync family | +| 7 | `GIT` | Server supports the `GIT_*` git introspection family | +| 8 | `LSP` | Server supports the `LSP_*` language intelligence family | +| 9 | `KV` | Server supports the `KV_*` key-value family | +| 10 | `NET` | Server supports the `NET_*` network-relay family | +| 11 | `EXTENSION` | Proposed: Wasmi extension lifecycle, events, and commands | +| 12 | `CHANNEL` | Proposed: server supports bidirectional named channels | +| 13 | `PROCESS` | Server supports native non-PTY child processes | +| 14 | `CREATE_STATUS` | `CREATE2(WANT_STATUS)` receives an explicit failure | +| 15 | `KILL_MODE` | `KILL`/`CLOSE` reach the process group; `KILL` takes `flags` | +| 16 | `PTY_DEADLINE` | `C2S_DEADLINE`, `CREATE2(HAS_DEADLINE)`, and `EXITED.reason` | +| 17 | `SCROLL_BY` | Scrollback holds still: `S2C_SCROLL_OFFSET` and `C2S_SCROLL_BY` | +| 18 | `SURFACE_TOUCH` | Server accepts direct contacts and exposes `wl_seat.touch` | +| 19 | `SURFACE_TEXT_INPUT` | Server forwards committed `zwp_text_input_v3` state | +| 20 | `CLIENT_CONTROL` | Enumerate connections and kick another client with a reason | +| 21 | `DESKTOP` | Compositor tray/notification state bridge and core API are live | +| 22 | `DESKTOP_MEDIA` | Viewer media, portals, and MPRIS control family is understood | +| 23 | `PROCESS_SESSION_ENV` | `PROCESS_SPAWN` accepts the `SESSION_ENV` flag (bit 2) | +| 24 | `ENV` | Server answers `ENV_GET` with its own environment | +| 25 | `APP_SOCKET` | `C2S_APP_SOCKET` mints per-application Wayland sockets | +| 26 | `CHANNEL_WATCH` | `CHANNEL_WATCH` follows which channel names have a listener | +| 27 | `CLIENT_ORIGIN` | The client catalog can say which connections are extensions | +| 28 | `TERM_JOURNAL` | Per-command journal and sequence-addressed output | + +Bit 26 is advertised with bit 12 and never alone; it is separate because a +`WATCH` an older server does not know is dropped by the channel family's +unknown-kind skip rule, which a client cannot tell from a name nobody serves. Bits 11 and 12 remain proposed for the extension and channel families under review in [#167](https://github.com/indent-com/blit/pull/167) and @@ -355,6 +400,9 @@ Bits 11 through 13 are independently omitted when `BLIT_EXT=0`, `BLIT_CHANNEL=0`, or `BLIT_PROCESS=0`; disabled-family requests are refused as specified in [design/extensions.md](design/extensions.md#security-posture-and-deployment-controls) and [design/processes.md](design/processes.md#security-and-deployment). +Bit 28 is omitted when `BLIT_TERM_JOURNAL=0`; nonce-bearing requests are +refused with `PERMISSION` as specified in +[design/term-journal.md](design/term-journal.md). Bit 14 is not extension-specific and is not controlled by those gates. It is advertised only after the server implements the negotiated creation outcome below; the implementation plan updates both shipped clients before enabling it. @@ -511,6 +559,38 @@ Two complementary paths report a PTY's working directory: Clients that predate `TERM_CWD_EVENT` are unaffected: consistent with the version-stability rule above (new message types are added under new opcodes), both reference clients drop unrecognized S2C opcodes — `js/core`'s `BlitConnection.handleMessage` dispatch falls through to a no-op `default`, and the CLI's message matches end in a catch-all `_ => {}`. +### Command journal + +A shell that emits OSC 133 (or OSC 633) brackets each command with markers. +The server turns those into a bounded ring of records per PTY and addresses +the output as a range of sequences — absolute grid-line indices that survive +scrollback eviction. Feature bit 28. Full contract in +[design/term-journal.md](design/term-journal.md); the emitting sequences live +in [shell-integration.md](shell-integration.md). + +- **List** (`C2S_TERM_JOURNAL` `0x50` → `S2C_TERM_JOURNAL` `0x50`): records + from `from_index`, or the last `limit` when `JOURNAL_TAIL` is set. + `oldest_index` is the lowest index still retained; `next_index` is what the + next command will take. +- **One command** (`C2S_TERM_OUTPUT` `0x51` → `S2C_TERM_OUTPUT` `0x51`): the + output region of `index`, capped by `max_bytes`. `index = u64::MAX` is the + newest record. +- **Since a cursor** (`C2S_TERM_SINCE` `0x52` → `S2C_TERM_OUTPUT` `0x51`): + everything appended after `(from_seq, from_col)`. `SINCE_PROBE` reports the + current cursor and returns no text. `next_seq`/`next_col` is what to send + back next time. `OUTPUT_TRUNCATED` means `max_bytes` cut the reply short; + `OUTPUT_EVICTED` means the start had already scrolled out; `OUTPUT_ALT_SCREEN` + means the PTY is on the alternate screen, where sequences do not advance. +- **Wait** (`C2S_TERM_JOURNAL_WAIT` `0x53` → `S2C_TERM_COMMAND` `0x52`): block + until `index` finishes. `index = u64::MAX` waits on whatever is running, or + on the next command to start if the shell is at a prompt. On timeout the + record comes back still flagged `RUNNING`. + +A terminal whose shell emits no markers keeps an empty journal. Older clients +that do not negotiate bit 28 never see these opcodes. + +Record on the wire: `[index:8][flags:1][exit_code:4][start_seq:8][end_seq:8][started_ms:8][ended_ms:8][cmd_len:2][command:N]`. + An opcode which multiplexes a one-byte inner kind also needs a family-defined skip rule. The proposed extension-command and native-channel families specify that clients ignore unknown S2C kinds, servers ignore unknown C2S kinds without diff --git a/docs/server.md b/docs/server.md index 5feb0d65..d252f950 100644 --- a/docs/server.md +++ b/docs/server.md @@ -10,6 +10,10 @@ | `SHELL` | `$SHELL` or `/bin/sh` | Shell spawned for new PTYs | | `BLIT_SHELL_FLAGS` | `li` (Unix) / `` (Windows) | Shell invocation flags | | `BLIT_SCROLLBACK` | `10000` | Scrollback buffer rows per PTY | +| `BLIT_TERM_JOURNAL` | `1` | `0` disables the OSC 133 command journal and its feature bit | +| `BLIT_TERM_JOURNAL_MAX` | `256` | Finished command records retained per PTY | +| `BLIT_TERM_JOURNAL_CMD_MAX` | `4096` | Bytes of command-line text retained per record | +| `BLIT_TERM_OUTPUT_MAX` | `1048576` (1 MiB) | Server ceiling on a `TERM_OUTPUT` / `TERM_SINCE` `max_bytes` | | `BLIT_VAAPI_DEVICE` | `/dev/dri/renderD128` | VA-API render node for surface encoding and camera decoding | | `BLIT_CUDA_DEVICE` | `0` | CUDA device ordinal for NVENC and NVDEC | | `BLIT_FD_CHANNEL` | unset | fd-channel file descriptor | @@ -475,7 +479,7 @@ graph LR | `wireplumber` | Minimal session manager (hardware monitors disabled) | | `pipewire-pulse` | PulseAudio compatibility socket | -Child processes inherit `PIPEWIRE_REMOTE` and `PULSE_SERVER` pointing at the private sockets. Monitor capture is handled in-process by `audio_pw::Capture` (`crates/server/src/audio_pw.rs`), which dlopens `libpipewire-0.3.so.0` at runtime and opens a capture stream directly on `blit-sink`'s monitor — no `pw-cat` subprocess, no pipe buffer, and the PipeWire quantum is set from client side so we don't inherit any third-party batching. The graph quantum is pinned to 1024/48000 (~21 ms, one Opus frame): the browser client sits behind a ≥ 60 ms jitter buffer so the extra batching is invisible, and the longer cycles give the graph threads 4× more scheduling slack when video encoding saturates the CPU. The daemon config loads `libpipewire-module-rt` (RT priority, nice -11 fallback) and the capture thread loop requests `loop.rt-prio`, so graph deadlines survive encode load even without RTKit. Audio availability is gated by `pipewire_available()` (checks for required binaries on PATH and for the libpipewire shared object being loadable) and can be disabled with `BLIT_AUDIO=0`. +Child processes inherit `PIPEWIRE_REMOTE` and `PULSE_SERVER` pointing at the private sockets. Monitor capture is handled in-process by `audio_pw::Capture` (`crates/server/src/audio_pw.rs`), which dlopens `libpipewire-0.3.so.0` at runtime and opens a capture stream directly on `blit-sink`'s monitor — no `pw-cat` subprocess, no pipe buffer, and the PipeWire quantum is set from client side so we don't inherit any third-party batching. The graph quantum is pinned to 1024/48000 (~21 ms, one Opus frame): the browser client sits behind a ≥ 60 ms jitter buffer so the extra batching is invisible, and the longer cycles give the graph threads 4× more scheduling slack when video encoding saturates the CPU. The daemon config loads `libpipewire-module-rt` (RT priority, nice -11 fallback) and the capture thread loop requests `loop.rt-prio`, so graph deadlines survive encode load even without RTKit. It also loads `libpipewire-module-profiler`, purely so those deadlines can be checked: `pw-top` and `pw-profiler` need the Profiler interface it registers and print nothing at all without it, so `XDG_RUNTIME_DIR=