chore: bump version to 0.6.0, update docs for API/background/Docker mounts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Author: PleasePrompto

AGENTS.md

@@ -4,7 +4,7 @@ This file gives coding agents a current map of the repository.
 
 ## Project Overview
 
-ductor is a Telegram bot that routes chat input to official provider CLIs (`claude`, `codex`, `gemini`), streams responses back to Telegram, persists per-chat state, and runs cron/webhook/heartbeat automation in-process. It also has an optional direct WebSocket API server (`api.enabled=true`) with authenticated file upload/download endpoints.
+ductor is a Telegram bot that routes chat input to official provider CLIs (`claude`, `codex`, `gemini`), streams responses back to Telegram, persists per-chat state, and runs cron/webhook/heartbeat automation in-process. It also supports `/bg` one-shot background tasks and an optional direct WebSocket API server (`api.enabled=true`) with authenticated file upload/download endpoints.
 
 Stack:
 
@@ -65,6 +65,7 @@ Direct API Message (optional)
 | `files/` | Shared file helpers (path tags, MIME detection, storage naming, media prompt builder) |
 | `orchestrator/` | command registry, directives/hooks, flow routing, observer wiring |
 | `cli/` | provider wrappers, stream parsing, auth checks, process registry, model caches |
+| `background/` | on-demand `/bg` task runner and async result delivery |
 | `session/` | chat sessions with provider-isolated buckets |
 | `cron/` | in-process scheduler and one-shot task execution |
 | `webhook/` | HTTP hooks (`wake` and `cron_task`) |
@@ -84,13 +85,14 @@ Direct API Message (optional)
 - Skill sync spans `~/.ductor/workspace/skills`, `~/.claude/skills`, `~/.codex/skills`, `~/.gemini/skills`.
   - normal mode: links
   - Docker mode: managed copies (`.ductor_managed` marker)
+- Docker user mounts (`docker.mounts`) map host directories into the sandbox under `/mnt/<name>`.
 - Streaming fallback is automatic; `/stop` abort checks are enforced during event loop processing.
 - Session state is provider-isolated; `/new` resets only the active provider bucket.
 - File access policy (`file_access`) is shared by Telegram file sends and API file downloads (`GET /files`).
 
 ## Background Systems
 
-All run as in-process asyncio tasks:
+Periodic in-process tasks:
 
 - `CronObserver`
 - `HeartbeatObserver`
@@ -102,6 +104,10 @@ All run as in-process asyncio tasks:
 - skill sync watcher
 - update observer (upgradeable installs)
 
+On-demand in-process subsystem:
+
+- `BackgroundObserver` (`/bg` task execution)
+
 Optional network service (not a periodic observer task):
 
 - `ApiServer`
@@ -128,6 +134,11 @@ Optional network service (not a periodic observer task):
 | `ductor docker rebuild` | Stop bot, remove container & image, rebuilt on next start |
 | `ductor docker enable` | Set `docker.enabled = true` |
 | `ductor docker disable` | Stop container, set `docker.enabled = false` |
+| `ductor docker mount <path>` | Add host directory mount to `docker.mounts` |
+| `ductor docker unmount <path>` | Remove host directory mount from `docker.mounts` |
+| `ductor docker mounts` | Show configured Docker mounts and targets |
+| `ductor api enable` | Enable direct WebSocket API server config (beta) |
+| `ductor api disable` | Disable direct WebSocket API server config (beta) |
 | `ductor service install` | Install as background service |
 | `ductor service [sub]` | Service management (status/stop/logs/...) |
 

README.md

@@ -33,6 +33,7 @@ You can:
 
 - chat from Telegram with Claude/Codex/Gemini,
 - stream responses live into edited Telegram messages,
+- offload one-off tasks via `/bg` and receive a final result when they finish,
 - run cron jobs and webhooks,
 - let heartbeat checks proactively notify you,
 - isolate runtime with Docker.
@@ -77,10 +78,11 @@ ductor executes the real provider CLIs as subprocesses. It does not proxy or spo
 
 ### Automation
 
+- Background tasks (`/bg`): push one-off tasks from the main chat into async execution while the chat stays responsive; receive a final completion message when done
 - Cron jobs: in-process scheduler with timezone support, per-job overrides, quiet hours, dependency queue
 - Webhooks: `wake` (inject into active chat) and `cron_task` (isolated task run) modes
 - Heartbeat: proactive checks in active sessions with cooldown + quiet hours
-- Cleanup: daily retention cleanup for `telegram_files/` and `output_to_user/`
+- Cleanup: daily retention cleanup for `telegram_files/`, `output_to_user/`, and `api_files/`
 
 ### Infrastructure
 
@@ -89,6 +91,7 @@ ductor executes the real provider CLIs as subprocesses. It does not proxy or spo
   - macOS: `launchd` Launch Agent
   - Windows: Task Scheduler task
 - Docker sidecar sandbox support (`Dockerfile.sandbox`)
+- Configurable Docker host-directory mounts (`docker.mounts`, `ductor docker mount ...`)
 - Restart protocol (`EXIT_RESTART = 42`) + sentinel-based user notifications
 - Version/update flow with Telegram `/upgrade` callback path
 
@@ -125,10 +128,28 @@ Detailed installation: [`docs/installation.md`](docs/installation.md)
 ductor docker enable
 ductor docker disable
 ductor docker rebuild
+ductor docker mount /absolute/or/env/path
+ductor docker unmount /absolute/or/env/path
+ductor docker mounts
 ```
 
 - `enable` / `disable`: toggles `docker.enabled` in `~/.ductor/config/config.json`
 - `rebuild`: stops the bot, removes Docker container + image, rebuilds on next start
+- `mount`: adds a host directory to `docker.mounts` (resolved absolute path; duplicates ignored)
+- `unmount`: removes a configured mount (exact/resolved/basename match)
+- `mounts`: shows configured host mounts and resolved container targets (`/mnt/<name>`)
+- mount/unmount changes require restart or container rebuild to apply
+
+## API management (beta)
+
+```bash
+ductor api enable
+ductor api disable
+```
+
+- `enable`: writes/updates `config.api`, generates token if missing
+- `disable`: sets `config.api.enabled=false`
+- restart required after changes
 
 ## Run as a background service
 
@@ -159,17 +180,10 @@ You (Telegram)
   -> streamed or non-streamed response back to Telegram
 ```
 
-Background observers run in the same process:
+Background systems run in the same process:
 
-- `CronObserver`
-- `HeartbeatObserver`
-- `WebhookObserver`
-- `CleanupObserver`
-- `CodexCacheObserver`
-- `GeminiCacheObserver`
-- `UpdateObserver` (only for upgradeable installs: `pipx`/`pip`)
-- rule sync watcher (`CLAUDE.md`/`AGENTS.md`/`GEMINI.md`)
-- skill sync watcher
+- periodic observers/watchers: `CronObserver`, `HeartbeatObserver`, `WebhookObserver`, `CleanupObserver`, `CodexCacheObserver`, `GeminiCacheObserver`, `UpdateObserver` (only for upgradeable installs), rule sync watcher, skill sync watcher
+- on-demand subsystem: `BackgroundObserver` (`/bg` task runner)
 
 Session behavior (important):
 
@@ -201,8 +215,11 @@ Session behavior (important):
       user_tools/
     telegram_files/
     output_to_user/
+    api_files/
 ```
 
+`workspace/api_files/` is created lazily on first API upload.
+
 ## Configuration
 
 Config file: `~/.ductor/config/config.json`
@@ -255,6 +272,8 @@ Full schema: [`docs/config.md`](docs/config.md)
 | `ductor restart` | Stop and re-exec bot |
 | `ductor upgrade` | Upgrade package and restart (non-dev mode) |
 | `ductor uninstall` | Remove bot data + uninstall package |
+| `ductor docker <enable|disable|rebuild|mount|unmount|mounts>` | Docker mode/config + user mount management |
+| `ductor api <enable|disable>` | Direct WebSocket API toggle (beta) |
 | `ductor service ...` | Service management (`install/status/start/stop/logs/uninstall`) |
 | `ductor help` | Help + status |
 

docs/README.md

@@ -14,7 +14,7 @@ ductor routes chat input to official provider CLIs (`claude`, `codex`, `gemini`)
 8. `docs/modules/files.md` -- shared file parsing/storage/prompt helpers.
 9. `docs/modules/cli.md` -- provider wrappers, stream parsing, process control.
 10. `docs/modules/workspace.md` -- `~/.ductor` seeding, rule deployment/sync, runtime notices.
-11. Remaining module docs (`session`, `cron`, `webhook`, `heartbeat`, `cleanup`, `infra`, `supervisor`, `security`, `logging`, `skill_system`).
+11. Remaining module docs (`background`, `session`, `cron`, `webhook`, `heartbeat`, `cleanup`, `infra`, `supervisor`, `security`, `logging`, `skill_system`).
 
 ## System in 60 Seconds
 
@@ -23,13 +23,14 @@ ductor routes chat input to official provider CLIs (`claude`, `codex`, `gemini`)
 - `ductor_bot/files/`: shared file tag parsing, MIME detection/classification, storage naming, transport-agnostic media prompt builder.
 - `ductor_bot/orchestrator/`: command dispatch, directives/hooks, normal + heartbeat flows, observer/server wiring.
 - `ductor_bot/cli/`: Claude/Codex/Gemini wrappers, stream-event normalization, process registry, auth detection, model caches.
+- `ductor_bot/background/`: on-demand `/bg` task execution and async result delivery.
 - `ductor_bot/session/`: per-chat session lifecycle with provider-isolated buckets in `sessions.json`.
 - `ductor_bot/cron/`: in-process scheduler for `cron_jobs.json` with task overrides, quiet hours, dependency queue.
 - `ductor_bot/webhook/`: HTTP ingress (`/hooks/{hook_id}`) with `bearer`/`hmac`, `wake`/`cron_task`, and shared dependency queue.
 - `ductor_bot/heartbeat/`: periodic proactive checks in active sessions.
-- `ductor_bot/cleanup/`: daily retention cleanup for top-level files in `telegram_files`, `output_to_user`, and `api_files`.
+- `ductor_bot/cleanup/`: daily recursive retention cleanup for `telegram_files`, `output_to_user`, and `api_files` (plus empty-dir pruning).
 - `ductor_bot/workspace/`: path resolution, home seeding from `_home_defaults`, RULES variant deployment, rule sync, skill sync.
-- `ductor_bot/infra/`: PID lock, restart/update sentinels, Docker manager, service backends (Linux/macOS/Windows), updater/version checks.
+- `ductor_bot/infra/`: PID lock, restart/update sentinels, Docker manager (incl. optional user mounts), service backends (Linux/macOS/Windows), updater/version checks.
 
 Runtime behavior note:
 
@@ -45,6 +46,7 @@ Runtime behavior note:
 - Module docs:
   - [setup_wizard](modules/setup_wizard.md)
   - [bot](modules/bot.md)
+  - [background](modules/background.md)
   - [api](modules/api.md)
   - [files](modules/files.md)
   - [cli](modules/cli.md)

docs/architecture.md

@@ -29,12 +29,13 @@ Direct API message (optional, `api.enabled=true`)
   -> WebSocket stream events + final result
 ```
 
-Also running in background:
+Background systems:
 
 - `CronObserver`: schedules `cron_jobs.json` entries.
 - `HeartbeatObserver`: periodic checks in existing sessions.
 - `WebhookObserver`: HTTP ingress for external triggers.
 - `CleanupObserver`: daily retention cleanup for workspace file directories.
+- `BackgroundObserver`: on-demand `/bg` task execution and result delivery.
 - `GeminiCacheObserver`: periodic Gemini model-cache refresh (`~/.ductor/config/gemini_models.json`).
 - `CodexCacheObserver`: periodic Codex model-cache refresh (`~/.ductor/config/codex_models.json`).
 - `UpdateObserver`: periodic PyPI version check + Telegram notification (upgradeable installs only).
@@ -57,7 +58,7 @@ Default path:
 4. Configure logging.
 5. Load/create `~/.ductor/config/config.json`.
 6. Deep-merge runtime config with `AgentConfig` defaults.
-7. Run `init_workspace(paths)`.
+7. Run `init_workspace(paths)` inside `load_config()`.
 8. Validate required config fields.
 9. Acquire PID lock (`bot.pid`, `kill_existing=True`).
 10. Start `TelegramBot`.
@@ -76,21 +77,18 @@ Default path:
 ### `Orchestrator.create()` (`ductor_bot/orchestrator/core.py`)
 
 1. Resolve paths from `ductor_home`.
-2. Run `init_workspace(paths)` in a worker thread.
-3. Set `DUCTOR_HOME` env var.
-4. If Docker enabled: run `DockerManager.setup()` and keep recovery wiring.
-5. If Docker container is active: re-sync skills in Docker-safe copy mode.
-6. Inject runtime environment notice into workspace rule files (`inject_runtime_environment`).
-7. Build orchestrator instance.
-8. Check provider auth (`check_all_auth`) and set authenticated provider set.
-9. Start `GeminiCacheObserver` (`~/.ductor/config/gemini_models.json`) and refresh runtime Gemini model registry from its callback.
-10. Start `CodexCacheObserver` (`~/.ductor/config/codex_models.json`).
-11. Create `CronObserver` and `WebhookObserver` with shared Codex cache.
-12. Start cron, heartbeat, webhook, cleanup observers.
-13. If `api.enabled=true`: start `ApiServer` (auto-generate token when empty, wire message/abort handlers and file context).
-14. Start rule-sync and skill-sync watcher tasks.
-
-`init_workspace()` is called in both `__main__.py` and `Orchestrator.create()`; behavior is idempotent.
+2. Set `DUCTOR_HOME` env var.
+3. If Docker enabled: run `DockerManager.setup()` (includes auth mounts + optional `mount_host_cache` + `docker.mounts`) and keep recovery wiring.
+4. If Docker container is active: re-sync skills in Docker-safe copy mode.
+5. Inject runtime environment notice into workspace rule files (`inject_runtime_environment`).
+6. Build orchestrator instance.
+7. Check provider auth (`check_all_auth`) and set authenticated provider set.
+8. Start `GeminiCacheObserver` (`~/.ductor/config/gemini_models.json`) and refresh runtime Gemini model registry from its callback.
+9. Start `CodexCacheObserver` (`~/.ductor/config/codex_models.json`).
+10. Create `BackgroundObserver`, `CronObserver`, and `WebhookObserver` (cron/webhook share Codex cache).
+11. Start cron, heartbeat, webhook, cleanup observers.
+12. If `api.enabled=true`: start `ApiServer` (auto-generate token when empty, wire message/abort handlers and file context).
+13. Start rule-sync and skill-sync watcher tasks.
 
 ## Message Routing
 
@@ -168,7 +166,8 @@ Per connection:
 
 1. `ws://<host>:<port>/ws` handshake.
 2. First frame must be auth JSON with token (10s timeout).
-3. Session `chat_id` defaults to first `allowed_user_ids` entry (fallback `1`); auth payload may override.
+   - `auth_ok` returns E2E server key plus provider metadata (`providers`) and active runtime fields when available.
+3. Session `chat_id` defaults to `config.api.chat_id` (`>0`) or first `allowed_user_ids` entry (fallback `1`); auth payload may override.
 4. `message` frames run under per-`chat_id` lock and use streaming callbacks (`text_delta`, `tool_activity`, `system_status`, `result`).
 5. `abort` frame or `/stop` message calls orchestrator abort path (`ProcessRegistry.kill_all`).
 
@@ -178,8 +177,6 @@ Additional HTTP endpoints:
 - `GET /files?path=...` (Bearer auth + `file_access` root checks),
 - `POST /upload` (Bearer auth + multipart save to `workspace/api_files/YYYY-MM-DD/`).
 
-Current wiring note: `config.api.chat_id` exists in schema but is not used by startup defaulting.
-
 ## Callback Query Flow
 
 `TelegramBot._on_callback_query()`:
@@ -201,6 +198,14 @@ Lock usage is path-dependent (e.g., queue cancel and upgrade callbacks are handl
 
 ## Background Systems
 
+### Background (`/bg`) flow
+
+- `TelegramBot._on_bg(...)` submits one-shot work via `Orchestrator.submit_background_task(...)`.
+- `BackgroundObserver` enforces max 5 active tasks per chat and runs tasks asynchronously.
+- Execution uses shared one-shot task runner (`infra/task_runner.py`) with current provider/model config.
+- Completion callback (`TelegramBot._on_bg_result`) sends a new Telegram message (notification-friendly).
+- `/stop` abort path cancels both active CLI subprocesses and active background tasks for the chat.
+
 ### Cron flow
 
 - `CronObserver` watches `cron_jobs.json` mtime every 5 seconds.
@@ -246,8 +251,8 @@ Lock usage is path-dependent (e.g., queue cancel and upgrade callbacks are handl
 
 - Hourly check in `user_timezone`.
 - Runs at most once per day when local hour equals `cleanup.check_hour`.
-- Deletes old top-level files in `workspace/telegram_files/`, `workspace/output_to_user/`, and `workspace/api_files/`.
-- Deletion is non-recursive, so files inside date subdirectories (`YYYY-MM-DD/`) are not cleaned by current logic.
+- Deletes old files recursively in `workspace/telegram_files/`, `workspace/output_to_user/`, and `workspace/api_files/`.
+- Prunes empty subdirectories after file deletion (including date-based upload folders).
 
 ## Restart & Supervisor
 

docs/automation.md

@@ -147,7 +147,7 @@ Default prompt asks the model to review memory + cron context and either send so
 
 Cleanup runs once per day at `cleanup.check_hour` (in `user_timezone`), checked hourly.
 
-Deletes old top-level files from:
+Deletes old files (recursive) from:
 
 - `workspace/telegram_files/`
 - `workspace/output_to_user/`
@@ -159,12 +159,7 @@ Retention windows:
 - `cleanup.output_to_user_days`
 - `cleanup.api_files_days`
 
-Cleanup is non-recursive.
-
-Current implication:
-
-- uploads are stored in dated subdirectories (`YYYY-MM-DD/`),
-- those subdirectory files are not removed by current cleanup logic.
+Cleanup also prunes empty subdirectories after deletion, so dated upload folders are removed once empty.
 
 ## Config blocks