Background task execution: non-blocking long-running terminal commands with completion notification (run_in_background parity)

[gildeshiro]

Summary

Piebald's terminal runs synchronously: a long-running command (build, test suite, multi-LLM debate, large download) blocks the chat until it returns — the user cannot send another message while it runs. Claude Code solves this with run_in_background (detached dispatch) plus a BashOutput stream and a completion event that surfaces the result back into the conversation. Piebald has no equivalent. This is a feature request for native background execution: dispatch a long command detached, keep chatting, and have the agent notified when it finishes.

Current behavior

• Every RunTerminalCommand is blocking. Firing cargo build --release (or a test suite, or a Gemini/Codex debate) freezes the conversation for the whole duration; the input box is effectively unusable until the command returns.
• There is no built-in way to (a) launch detached, (b) poll a running job's output, or (c) be told a job finished.

Proposed solution

1. A background flag on the terminal tool (e.g. run_in_background: true) that returns immediately with a job handle instead of blocking.
2. Job introspection tools — list jobs, read a job's accumulated stdout/stderr, and kill a job (parity with Claude Code's BashOutput / KillShell).
3. A completion-surfacing mechanism. Claude Code injects completion mid-loop. Piebald has no event loop that injects a message into a turn without user input, so even a "pull on next user prompt" surfacing (inject the completion notice at the start of the next turn) would be a large improvement and fits Piebald's current model.

Why this matters

Long, non-blocking work (builds, test runs, multi-LLM orchestration, downloads) is routine in agentic development. Today each one forces a hard stall on the chat. Background execution restores the conversational flow that makes the agent usable during long operations.

Reference implementation (userland workaround)

I built a working approximation, piebald-bgrun — a trio that emulates run_in_background on top of hooks + app.db:
https://github.com/gildeshiro/piebald-bgrun

• A — Trigger: a directive appended to the profile's system prompt (the reflex of when to background).
• B — Friction: bgrun / bg-status / bg-kill wrappers that dispatch detached (Start-Process on Windows, setsid on Linux) and track job state under ~/.piebald-bg/<job>/.
• C — Wake: a UserPromptSubmit hook that scans for finished jobs and injects a <background-task-status> block on the next turn (idempotent via an .ack sentinel).

It works well in practice (dispatch → keep chatting → status appears next turn), but it is a workaround: it depends on the next user prompt to surface completion, and it edits app.db / wires hooks rather than using a first-class API.

Limitation this can't solve from userland

Piebald has no loop to push a message mid-turn without user input, so my hook wakes on the next prompt (pull), not push mid-generation like Claude Code. A native implementation could close this gap and make the notification immediate.

Environment

• Piebald: 0.4.4
• OS: Windows 11

[gildeshiro]

Update — userland piebald-bgrun now pushes to the origin chat, plus a cross-client render limitation worth surfacing

Since filing this, I extended the userland approximation from pull-on-next-turn to an actual push: on completion it resolves the originating chat and injects the recap into that chat via the web WS API (send_message_streaming, through the piebald-web bridge), so the loop continues in the right chat without waiting for the next user prompt. Two findings relevant to a native implementation:

1. Chat identity isn't exposed to the terminal. A detached job can't tell which chat fired it — TERM_PROGRAM=piebald is the only env hint, the UserPromptSubmit hook payload carries no chat id, and the terminal tool's tool_input only lands in app.db after the command returns. I resolve the origin by matching the launch command in message_part_tool_call.tool_input nearest the job's start time. A native feature could just hand the job its chat_id.

2. Cross-client live-render limitation (the big one). Anything that surfaces a completion by injecting a message/generation into a chat — native or userland — hits this: a generation initiated by one connection does not render live on other clients viewing the same chat; they only catch up on refresh.

• Transport is fine — the engine broadcasts StreamedChunk to every WS connection. A separate passive WS that initiated nothing still received the full live stream (19 chunks for a chat driven by another connection; 72 for another chat).
• The gate is client-side: the web client (and the desktop app, which hosts the same React bundle) applies an incoming chunk to the open thread only if that chat's messages are already in its query cache — effectively setQueryData(messages(chatId), e => e ? applyChunk(e) : (refetch(), e)). Chats not currently loaded drop the live chunks, so a background-completion push lands correctly but stays invisible until that surface is refreshed.

For a first-class background feature, (2) is what makes the completion actually visible: ideally clients would apply (or warm-load then apply) broadcast generation chunks for a chat even when it isn't the focused/cached one, rather than refetch-on-refresh. Happy to share the reverse-engineering notes if useful.