diff --git a/.gitignore b/.gitignore index 13ff498..e4c58ff 100644 --- a/.gitignore +++ b/.gitignore @@ -14,3 +14,4 @@ benchmarks/results/* !benchmarks/results/20260327-111817-https_www.google.com_travel_flights_q=Flights%20to%20TPE%20from%/ !benchmarks/results/20260327-111817-https_www.google.com_travel_flights_q=Flights%20to%20TPE%20from%/** kuri-browser/PARITY.md +skills/pi-kuri-plugin/package-lock.json diff --git a/readme.md b/readme.md index 42c043c..ddde92d 100644 --- a/readme.md +++ b/readme.md @@ -502,6 +502,9 @@ The lowest-friction server loop is: The repo includes a user-extensible skill area: - `skills/kuri-skill.md` is the base Kuri HTTP-agent skill +- `skills/pi-kuri-skills.md` is a two-tier pi.dev skill (core operations + compact advanced hint) +- `skills/pi-kuri-advanced.md` is the companion advanced reference for pi.dev (click, type, scroll, JS eval, cookies, audit, HAR, etc.) +- `skills/pi-kuri-plugin/` is a standalone npm-installable pi.dev plugin with CLI wrapper (`kuri-skill`), package.json, and skill files - `skills/custom/` is reserved for your own project-specific skills - `skills/custom/hackernews-page-2.md` is a concrete example custom skill - `.claude/skills/kuri-server/SKILL.md` stays in sync for Claude-style repo skills diff --git a/skills/pi-kuri-advanced.md b/skills/pi-kuri-advanced.md new file mode 100644 index 0000000..8d9ff81 --- /dev/null +++ b/skills/pi-kuri-advanced.md @@ -0,0 +1,354 @@ +--- +name: pi-kuri-advanced +description: Advanced Kuri browser automation for pi.dev agents โ€” click, type, fill, select, scroll, JavaScript evaluation, cookies, security audits, session management, HAR recording, and the experimental kuri-browser CLI. Load this on demand when the core skill (`skills/pi-kuri-skills.md`) is not enough. +--- + +# Pi Kuri Advanced Reference + +This file covers operations beyond the core skill (`skills/pi-kuri-skills.md`). +Load it when you need interactive actions, JavaScript evaluation, cookies, +security audits, session management, or experimental features. + +All commands use the `kuri-skill` CLI from `skills/pi-kuri-plugin/`. If +`kuri-skill` is not on your PATH, run commands via: + +```bash +node skills/pi-kuri-plugin/scripts/kuri.js +``` + +Set `KURI_SESSION` to group tabs without repeating `tab_id` on every call: + +```bash +export KURI_SESSION=my-session +``` + +--- + +## ๐Ÿ–ฑ๏ธ Mouse Actions + +### Click an element + +```bash +kuri-skill action click e1 +kuri-skill action click ref=e1 tab_id=TABID123 + +# Right-click +kuri-skill action click ref=e1 button=right + +# Double-click +kuri-skill action click ref=e1 clickCount=2 +``` + +### Hover + +Use the snapshot to detect hover-triggered UI, then evaluate JS for precise +hover scenarios: + +```bash +kuri-skill action evaluate \ + expression="document.querySelector('button').dispatchEvent(new MouseEvent('mouseover',{bubbles:true}))" +``` + +--- + +## โŒจ๏ธ Keyboard Actions + +### Type text into an input + +```bash +kuri-skill action type e2 "hello world" +kuri-skill action type ref=e2 value="hello world" +``` + +### Fill input value + +```bash +kuri-skill action fill e2 "user@example.com" +kuri-skill action fill ref=e2 value="user@example.com" +``` + +### Select a dropdown option + +```bash +kuri-skill action select e3 "option-value" +kuri-skill action select ref=e3 value="option-value" +``` + +--- + +## ๐Ÿ“œ Scrolling + +```bash +# Scroll down (default) +kuri-skill action scroll + +# Scroll up +kuri-skill action scroll direction=up amount=10 + +# Scroll right +kuri-skill action scroll direction=right amount=3 + +# Scroll by pixel amount +kuri-skill action scroll direction=down amount=500 +``` + +--- + +## ๐Ÿง  JavaScript Evaluation + +```bash +# Get page title +kuri-skill action evaluate expression="document.title" + +# Read heading text +kuri-skill action evaluate \ + expression="document.querySelector('h1').textContent" + +# Extract dynamic state +kuri-skill action evaluate \ + expression="JSON.stringify(window.__INITIAL_STATE__)" + +# Get page metrics +kuri-skill action evaluate \ + expression="document.body.scrollHeight" + +# Check element visibility +kuri-skill action evaluate \ + expression="document.querySelector('.spinner').offsetParent!==null" +``` + +Returns the evaluated result as a string, number, boolean, or JSON value. + +--- + +## ๐Ÿช Cookies + +```bash +kuri-skill action cookies +``` + +Returns an array of cookies with: `name`, `value`, `domain`, `path`, `secure`, +`httpOnly`, `sameSite`, `expires`. + +--- + +## ๐Ÿ”’ Security Audit + +```bash +kuri-skill action audit +``` + +Checks performed: +- Security response headers: HSTS, CSP, X-Frame-Options, X-Content-Type-Options +- Cookie security: Secure flag, HttpOnly flag, SameSite attribute +- HTTPS enforcement +- Mixed content warnings + +--- + +## ๐Ÿ“ก HAR Recording (HTTP Archive) + +Record network activity for API discovery or debugging: + +```bash +# Start recording +kuri-skill action har-start + +# Perform actions that make network requests... + +# Stop and retrieve the HAR log +kuri-skill action har-stop + +# Get current HAR log without stopping +kuri-skill action har +``` + +Useful for reverse-engineering API calls or debugging network issues. + +--- + +## ๐Ÿ“‘ Tab & Session Management + +```bash +# List all tabs +kuri-skill tabs + +# Create a new tab +kuri-skill tab-new https://example.com + +# Close a tab +kuri-skill action close-tab TABID123 + +# Browser back +kuri-skill action back + +# Browser forward +kuri-skill action forward + +# Reload +kuri-skill action reload +``` + +### Multiple sessions + +Run independent browser sessions by setting `KURI_SESSION` to different values: + +```bash +KURI_SESSION=session-a kuri-skill tab-new https://example.com +KURI_SESSION=session-b kuri-skill tab-new https://other.com +``` + +Each session tracks its own tabs and current tab independently. + +--- + +## ๐Ÿงช Experimental โ€” kuri-browser + +The `kuri-browser/` subdirectory is an **experimental** Zig-native browser +runtime with its own rendering engine (separate build, not wired into Kuri's +root build). Use for evaluation and development only. + +```bash +cd kuri-browser +zig build run -- render https://news.ycombinator.com --selector ".titleline a" --dump text +zig build run -- render https://todomvc.com/examples/react/dist/ --js --wait-eval "..." +zig build run -- bench --offline +zig build run -- serve-cdp --port 9333 +``` + +Screenshots fall back to the main Kuri/CDP renderer โ€” start the normal Kuri +server first, then: + +```bash +cd kuri-browser +zig build run -- screenshot https://example.com --out example.jpg \ + --compress --kuri-base http://127.0.0.1:8080 +``` + +--- + +## Common Workflow Patterns + +### Pattern 1: Quick screenshot verification + +```bash +export KURI_SESSION=verify + +kuri-skill tab-new https://example.com +kuri-skill page-info +kuri-skill screenshot # set KURI_OUTPUT to customize path +``` + +### Pattern 2: Fill form and submit + +```bash +export KURI_SESSION=login + +kuri-skill tab-new https://example.com/login +kuri-skill snap # Get refs + +kuri-skill action type e0 "user@example.com" +kuri-skill action type e1 "password123" +kuri-skill action click e2 # Submit button + +kuri-skill page-info # Check result +``` + +### Pattern 3: Extract dynamic content + +```bash +export KURI_SESSION=extract + +kuri-skill tab-new https://example.com/app +kuri-skill action evaluate expression="JSON.stringify(window.__DATA__)" +kuri-skill text # Also get rendered text +``` + +### Pattern 4: Multi-step interaction loop + +```bash +export KURI_SESSION=search + +# 1. Open page +kuri-skill tab-new https://example.com/search + +# 2. Get snapshot for refs +kuri-skill snap + +# 3. Type search query +kuri-skill action type e0 "query text" + +# 4. Check for results via JS +kuri-skill action evaluate \ + expression="document.querySelectorAll('.result').length" + +# 5. Re-snap after DOM change +kuri-skill snap + +# 6. Click a result +kuri-skill action click e3 + +# 7. Verify new page +kuri-skill page-info + +# 8. Screenshot for evidence +kuri-skill screenshot +``` + +--- + +## Quick Parameter Reference + +| Parameter | Applies to | Description | +|-----------|-----------|-------------| +| `ref` | action | Element reference from snapshot | +| `action` | action | `click`, `type`, `fill`, `select`, `scroll` | +| `value` | type / fill / select | Text or option value | +| `direction` | scroll | `up`, `down`, `left`, `right` | +| `amount` | scroll | Scroll distance (lines or pixels) | +| `button` | click | `left`, `right`, `middle` | +| `clickCount` | click | `1` = single, `2` = double | +| `modifiers` | click | `shift`, `ctrl`, `meta` | +| `expression` | evaluate | JavaScript to execute | +| `tab_id` | Most commands | Target a specific tab | + +## Full Endpoint Map + +The `kuri-skill` CLI wraps all major Kuri HTTP endpoints. For the complete +list of ~80+ endpoints, see the main Kuri README. + +| Endpoint | `kuri-skill` | Purpose | +|----------|-------------|---------| +| `/health` | `health` | Server health check | +| `/tabs` | `tabs` | List all tabs | +| `/tab/new` | `tab-new` | Create a new tab | +| `/tab/close` | `action close-tab` | Close a tab | +| `/tab/current` | โ€” (use `KURI_SESSION`) | Set current tab | +| `/navigate` | `navigate` | Navigate to URL | +| `/page/info` | `page-info` | Page metadata | +| `/page/state` | โ€” | Lightweight observation | +| `/screenshot` | `screenshot` | Capture screenshot | +| `/snapshot` | `snap` | Accessibility tree | +| `/text` | `text` | Page plain text | +| `/markdown` | `markdown` | Page as markdown | +| `/links` | `links` | Extract links | +| `/action` | `action ` | Click, type, fill, select, scroll | +| `/evaluate` | `action evaluate` | Execute JavaScript | +| `/cookies` | `action cookies` | List cookies | +| `/audit` | `action audit` | Security audit | +| `/back` | `action back` | Browser back | +| `/forward` | `action forward` | Browser forward | +| `/reload` | `action reload` | Reload page | +| `/har` | `action har` | Current HAR log | +| `/har/start` | `action har-start` | Start HAR recording | +| `/har/stop` | `action har-stop` | Stop HAR recording | +| `/batch` | โ€” | Multi-command batch | +| `/evaluate` | `action evaluate` | JS evaluation | +| `/clipboard/*` | โ€” | Clipboard read/write | +| `/dialog/*` | โ€” | Dialog handling | +| `/mouse/*` | โ€” | Mouse control | +| `/recording/*` | โ€” | Action recording | +| `/vitals` | โ€” | Core Web Vitals | +| `/cache/*` | โ€” | Action caching | + +For the complete list see the main [Kuri README](../readme.md). diff --git a/skills/pi-kuri-plugin/.npmignore b/skills/pi-kuri-plugin/.npmignore new file mode 100644 index 0000000..7a7b33a --- /dev/null +++ b/skills/pi-kuri-plugin/.npmignore @@ -0,0 +1,2 @@ +package-lock.json +node_modules diff --git a/skills/pi-kuri-plugin/OWNER.md b/skills/pi-kuri-plugin/OWNER.md new file mode 100644 index 0000000..8acd4e6 --- /dev/null +++ b/skills/pi-kuri-plugin/OWNER.md @@ -0,0 +1,234 @@ +# pi-kuri-skill โ€” Maintainer's Guide + +This document covers how to publish and maintain `pi-kuri-skill` as a +[pi.dev](https://pi.dev) skill package on npm. + +## Quick Reference + +| What | Where | +|------|-------| +| Package directory | `skills/pi-kuri-plugin/` | +| npm package name | `pi-kuri-skill` | +| Install command | `pi install npm:pi-kuri-skill` | +| pi.dev catalog | [pi.dev/packages](https://pi.dev/packages) (auto-discovered from npm) | +| Example packages | `pi install npm:pi-subagents`, `pi install npm:pi-web-access` | + +## Directory Layout + +``` +skills/pi-kuri-plugin/ +โ”œโ”€โ”€ package.json # npm manifest โ€” name, version, pi manifest, bin +โ”œโ”€โ”€ pi-kuri.ts # Pi extension โ€” registers all Kuri tools for agents +โ”œโ”€โ”€ SKILL.md # Pi agent skill (two-tier: core + advanced) +โ”œโ”€โ”€ README.md # User-facing install/config docs +โ”œโ”€โ”€ scripts/ +โ”‚ โ””โ”€โ”€ kuri.js # CLI binary โ€” kuri-skill command +โ”œโ”€โ”€ references/ +โ”‚ โ””โ”€โ”€ ADVANCED.md โ†’ # Symlink โ†’ ../../pi-kuri-advanced.md (canonical source) +โ””โ”€โ”€ .npmignore # Excludes package-lock.json, node_modules +``` + +## How Publishing Works + +pi.dev auto-discovers packages from the npm registry. When you publish (or +update) an npm package with the `pi-package` keyword and a `pi` manifest in +`package.json`, it appears on the [pi.dev package catalog](https://pi.dev/packages) +within minutes. No separate submission is needed. + +### What pi.dev looks for in `package.json` + +```json +{ + "name": "pi-kuri-skill", + "version": "1.0.0", + "keywords": ["pi-package", "agent-skills", "kuri", "browser-automation"], + "pi": { + "extensions": ["./pi-kuri.ts"], โ† registers Kuri tools + "skills": ["."] โ† registers SKILL.md + } +} +``` + +The `pi-package` keyword is the **discovery signal**. Without it, the package +won't appear on the catalog. + +For gallery previews, you can add optional metadata: + +```json +{ + "pi": { + "extensions": ["./pi-kuri.ts"], + "skills": ["."], + "image": "https://kuri.gg/screenshot.png", + "video": "https://kuri.gg/demo.mp4" + } +} +``` + +### Extension auto-registration + +Because `package.json` declares `"extensions": ["./pi-kuri.ts"]`, running +`pi install npm:pi-kuri-skill` automatically registers all Kuri tools +(`kuri_navigate`, `kuri_snap`, `kuri_screenshot`, etc.) โ€” no manual symlink +or copy step needed. + +## Step-by-Step: Publish a Release + +### 1. Bump the version + +```bash +cd skills/pi-kuri-plugin + +# Choose one: +npm version patch # 1.0.0 โ†’ 1.0.1 (bug fixes) +npm version minor # 1.0.0 โ†’ 1.1.0 (new features, backward-compatible) +npm version major # 1.0.0 โ†’ 2.0.0 (breaking changes) +``` + +This updates `package.json`, creates a git tag, and commits. + +### 2. Smoke-test the package + +```bash +# Verify the CLI works +node scripts/kuri.js health + +# Verify the package tarball is clean +npm pack --dry-run +``` + +Check that `npm pack --dry-run` lists only the files you intend to ship. +The `.npmignore` file excludes `package-lock.json` and `node_modules`. + +### 3. Push the tag + +```bash +git push --tags +``` + +### 4. Publish to npm + +```bash +cd skills/pi-kuri-plugin +npm publish +``` + +> **Note**: You need npm publish access for the `pi-kuri-skill` package name. +> If you haven't published before, run `npm login` first and ensure the name +> is available (or use a scoped name like `@your-org/pi-kuri-skill`). + +### 5. Verify on pi.dev + +```bash +# Install from npm in a test project +pi install npm:pi-kuri-skill + +# Check it's listed +pi list +``` + +If the package is listed, it will appear on [pi.dev/packages](https://pi.dev/packages) +within a few minutes. + +## Maintaining the Skill + +### Keeping docs in sync + +The two-tier skill documentation lives at two levels: + +| File | Purpose | +|------|---------| +| `skills/pi-kuri-skills.md` | Canonical core skill docs (repo-level) | +| `skills/pi-kuri-advanced.md` | Canonical advanced reference (repo-level) | +| `skills/pi-kuri-plugin/SKILL.md` | Plugin skill entrypoint โ€” references both tiers | +| `skills/pi-kuri-plugin/references/ADVANCED.md` | **Symlink** โ†’ `../../pi-kuri-advanced.md` โ€” always in sync | + +The symlink at `references/ADVANCED.md` ensures the npm package always ships +the same advanced reference as the repo-level canonical file. **Do not replace +it with a copy** โ€” that's how drift happens. + +### Updating the Pi extension + +The extension file `pi-kuri.ts` registers Kuri tools for the Pi agent. When +you add, remove, or change tool signatures: + +1. Update `pi-kuri.ts` +2. Update `SKILL.md` if the user-facing instructions change +3. Bump a patch version and publish + +### Version convention + +| Change | Version bump | Example | +|--------|-------------|---------| +| Bug fix, docs, tool description | `patch` | 1.0.0 โ†’ 1.0.1 | +| New tool, new parameter, backward-compatible | `minor` | 1.0.0 โ†’ 1.1.0 | +| Breaking tool API change, removed tool | `major` | 1.0.0 โ†’ 2.0.0 | + +## Example: Other pi.dev Packages for Reference + +Browse the [pi.dev package catalog](https://pi.dev/packages) for real examples. +Notable packages with similar structure (skill + CLI/extension): + +| Package | Type | What to learn | +|---------|------|---------------| +| `pi-subagents` | extension | Multi-file extension with TUI, well-structured `package.json` | +| `pi-web-access` | extension | Tool registration pattern, multi-tool extension | +| `pi-mcp-adapter` | extension | Extension with MCP protocol integration | +| `pi-package-search` | skill | Skill-only package, `/skill:` invocation pattern | +| `@juicesharp/rpiv-*` | extension | Scoped package + comprehensive test suite | + +Install any of them to inspect their structure: + +```bash +pi install npm:pi-subagents +pi install npm:pi-web-access +pi install npm:pi-mcp-adapter +pi install npm:pi-package-search + +# Then inspect: +cat ~/.pi/agent/npm/node_modules/pi-subagents/package.json +``` + +## Troubleshooting + +### Package not appearing on pi.dev + +1. Check that `keywords` in `package.json` includes `"pi-package"` +2. Verify the `pi` manifest section is present +3. Confirm the package is publicly accessible on npm +4. Wait a few minutes โ€” the catalog refreshes periodically + +### `pi install npm:pi-kuri-skill` fails + +```bash +# Try with explicit version +pi install npm:pi-kuri-skill@1.0.0 + +# Check npm for the package +npm view pi-kuri-skill + +# Install from local path for testing +pi install ./skills/pi-kuri-plugin +``` + +### Symlink broken after clone + +The `references/ADVANCED.md` symlink targets `../../pi-kuri-advanced.md` +relative to the plugin directory. On a fresh clone, verify: + +```bash +ls -la skills/pi-kuri-plugin/references/ADVANCED.md +readlink -f skills/pi-kuri-plugin/references/ADVANCED.md +# Should resolve to: /path/to/kuri/skills/pi-kuri-advanced.md +``` + +If the symlink is broken, re-create it: + +```bash +rm skills/pi-kuri-plugin/references/ADVANCED.md +ln -s ../../pi-kuri-advanced.md skills/pi-kuri-plugin/references/ADVANCED.md +``` + +> **For Windows users**: Git may not handle symlinks by default. Enable them +> with `git config --global core.symlinks true` before cloning, or replace +> the symlink with a copy script in a prepublish step. diff --git a/skills/pi-kuri-plugin/README.md b/skills/pi-kuri-plugin/README.md new file mode 100644 index 0000000..336db05 --- /dev/null +++ b/skills/pi-kuri-plugin/README.md @@ -0,0 +1,153 @@ +# pi-kuri-skill + +A [Pi](https://pi.dev) skill plugin for **Kuri** browser automation โ€” navigate, +screenshot, extract page content, and interact with web pages via Kuri's HTTP API. + +This package provides: + +- **`pi-kuri.ts`** โ€” Pi agent extension registering Kuri tools (`kuri_navigate`, + `kuri_snap`, `kuri_screenshot`, `kuri_text`, `kuri_evaluate`, `kuri_click`, + `kuri_console_errors`, etc.) for agent-driven browser automation +- **`kuri-skill` CLI** โ€” a portable Node.js wrapper around Kuri's HTTP API +- **`SKILL.md`** โ€” pi.dev agent skill with two-tier progressive disclosure +- **`references/ADVANCED.md`** โ€” on-demand reference for click, type, JS eval, etc. + +## Prerequisites + +- [Kuri](https://github.com/justrach/kuri) โ€” the Zig-native browser automation server +- Node.js 18+ + +## Install + +### As a pi.dev skill package + +```bash +pi install npm:pi-kuri-skill +``` + +### Standalone (global CLI) + +```bash +npm install -g pi-kuri-skill +``` + +### Extension setup + +That's it. `pi install` handles everything โ€” it registers the extension (tools like +`kuri_navigate`, `kuri_snap`, `kuri_screenshot`, etc.) and the skill (`SKILL.md`) +in one step. Restart pi.dev and the agent will have all Kuri tools available. + +Then use the `kuri-skill` command: + +```bash +kuri-skill health +kuri-skill navigate https://example.com +kuri-skill screenshot +``` + +### Project-local + +```bash +npm install pi-kuri-skill +npx kuri-skill health +``` + +### From the kuri repository + +```bash +cd skills/pi-kuri-plugin +npm install +npm link # makes `kuri-skill` available globally +``` + +## Quick Start + +1. **Start Kuri** (if not already running): + + ```bash + kuri + ``` + +2. **Check the server is alive:** + + ```bash + kuri-skill health + ``` + +3. **Navigate to a page:** + + ```bash + kuri-skill navigate https://example.com + ``` + +4. **Inspect the page:** + + ```bash + kuri-skill page-info + kuri-skill screenshot + kuri-skill text + ``` + +5. **Use interactive refs:** + + ```bash + kuri-skill snap + kuri-skill action click e1 + ``` + +## Core Commands + +| Command | Description | +|---------|-------------| +| `kuri-skill health` | Server health check | +| `kuri-skill tabs` | List all browser tabs | +| `kuri-skill tab-new [url]` | Open new tab | +| `kuri-skill navigate ` | Navigate to URL | +| `kuri-skill page-info` | Current page info | +| `kuri-skill screenshot` | Capture screenshot | +| `kuri-skill text` | Extract plain text | +| `kuri-skill markdown` | Extract as markdown | +| `kuri-skill links` | Extract all links | +| `kuri-skill snap` | Accessibility snapshot | +| `kuri-skill action <...>` | Advanced actions | +| `kuri-skill advanced` | Print advanced reference path | + +## Advanced Reference + +```bash +kuri-skill advanced +``` + +Or read `references/ADVANCED.md` directly for the full API reference covering +~80 endpoints, options, and workflow patterns. + +## Configuration + +| Variable | Default | Description | +|----------|---------|-------------| +| `KURI_BASE_URL` | `http://127.0.0.1:8080` | Kuri server URL | +| `KURI_SESSION` | `pi-kuri-skill` | Active session ID | +| `KURI_API_TOKEN` | (from `~/.kuri/api.token`) | API auth token | +| `KURI_TAB_ID` | (empty) | Default tab ID override | +| `KURI_OUTPUT` | `/tmp/kuri-*.png` | Screenshot output path | +| `KURI_PORT` | `8080` | Server port | + +## Structure + +``` +pi-kuri-plugin/ +โ”œโ”€โ”€ SKILL.md # Pi agent skill (loaded by pi) +โ”œโ”€โ”€ package.json # npm package metadata + pi.skills +โ”œโ”€โ”€ README.md # This file +โ”œโ”€โ”€ scripts/ +โ”‚ โ””โ”€โ”€ kuri.js # CLI wrapper (also the `kuri-skill` bin) +โ””โ”€โ”€ references/ + โ””โ”€โ”€ ADVANCED.md # On-demand advanced reference +``` + +## Related + +- [Kuri](https://github.com/justrach/kuri) โ€” Zig-native browser automation server +- [Pi](https://pi.dev) โ€” The coding agent harness +- [`skills/pi-kuri-skills.md`](../pi-kuri-skills.md) โ€” Canonical curl-based core skill +- [`skills/pi-kuri-advanced.md`](../pi-kuri-advanced.md) โ€” Canonical curl-based advanced reference diff --git a/skills/pi-kuri-plugin/SKILL.md b/skills/pi-kuri-plugin/SKILL.md new file mode 100644 index 0000000..f569651 --- /dev/null +++ b/skills/pi-kuri-plugin/SKILL.md @@ -0,0 +1,194 @@ +--- +name: pi-kuri +description: Kuri browser automation for pi.dev agents โ€” navigate, screenshot, extract content, and interact with web pages via the Kuri HTTP server. Use when an agent needs to browse the web, read pages, take screenshots, or perform browser interactions. This is the pi.dev plugin skill โ€” see skills/pi-kuri-skills.md for the canonical reference. +--- + +# Pi Kuri Plugin + +Kuri is a Zig-native browser automation server. This plugin provides: +- A Pi agent extension (`pi-kuri.ts`) with registered tools for agent-driven + browser automation โ€” navigate, screenshot, snap, click, type, evaluate, etc. +- A CLI wrapper (`kuri-skill`) for human use in the terminal +- On-demand reference docs for both tiers + +### Two Ways to Use Kuri + +| Method | What it gives you | When to use | +|--------|-------------------|-------------| +| **Pi extension** ๐Ÿง  | Registered tools: `kuri_navigate`, `kuri_snap`, `kuri_screenshot`, `kuri_text`, `kuri_evaluate`, `kuri_click`, `kuri_console_errors`, etc. | The agent drives the browser directly via tool calls | +| **CLI** โŒจ๏ธ | `kuri-skill` command: navigate, screenshot, text, markdown, snap, click, type | Human-in-the-loop or scripts | + +The extension is the primary interface for pi.dev agents. The CLI is for +manual use, debugging, and scripting. + +### Install the Extension + +The package registers itself. `pi install npm:pi-kuri-skill` handles the +extension โ€” no manual symlink needed. Restart pi.dev and the agent will +have all Kuri tools available. + +### Getting `kuri-skill` on PATH + +```bash +# From the plugin directory (when installed from npm): +npm link + +# Or run directly: +node path/to/scripts/kuri.js + +# When installed globally: +npm install -g pi-kuri-skill +``` + +This skill has two tiers: + +| Tier | Covered here | When to use | +|------|-------------|-------------| +| **Core** ๐ŸŸข | Server health check, start, navigate, screenshot, page info, text & markdown extraction, links, accessibility snapshots | Most tasks | +| **Advanced** ๐Ÿ”ต | Click, type, fill, select, scroll, JS eval, cookies, audit, HAR, session mgmt | Run `kuri-skill advanced` or read `skills/pi-kuri-advanced.md` | + +> **Direct HTTP alternative:** If you don't have Node.js, all operations work +> via direct HTTP calls โ€” see `skills/pi-kuri-skills.md` for details. + +--- + +## 1. Setup โ€” Validate & Start + +### Check if Kuri is running + +```bash +kuri-skill health +``` + +A healthy server returns `{"ok":true}` with the server version. + +If the health check fails โ€” connection refused or timeout โ€” the server is not running. + +### Start Kuri + +If you built from source: + +```bash +# From the kuri repo root +zig build +./zig-out/bin/kuri +``` + +If you installed via the install script: + +```bash +kuri +``` + +Kuri starts a Chrome instance (managing it automatically) and listens on +`http://127.0.0.1:8080` by default. Customize with `HOST`, `PORT`, `CDP_URL` +(see the configuration table in the README). + +### List available tabs + +```bash +kuri-skill tabs +``` + +Returns the active session's tabs with their IDs, URLs, and titles. + +--- + +## 2. Core Operations + +All examples use the `kuri-skill` CLI. + +### Create a tab and navigate + +```bash +kuri-skill tab-new https://example.com +kuri-skill navigate https://example.com +``` + +### Get page info + +```bash +kuri-skill page-info +``` + +Returns `url`, `title`, and page `readyState`. Always call this after navigation +to confirm the page loaded before taking further actions. + +### Screenshot + +```bash +kuri-skill screenshot +``` + +Saves a PNG. Customize the output path with `KURI_OUTPUT`. + +### Content extraction + +```bash +kuri-skill text +kuri-skill markdown +kuri-skill links +``` + +### Accessibility snapshot (interactive refs) + +```bash +kuri-skill snap +``` + +Returns a compact text-tree of interactive elements with refs like `e0`, `e1`. +These refs are used as targets for advanced actions (click, type, fill, etc.). + +**Re-snap after any navigation or DOM change** โ€” refs are snapshot-local and +become stale once the page updates. + +--- + +## 3. Advanced Operations + +For click, type, fill, select, scroll, JavaScript evaluation, cookies, security +audits, HAR recording, and session management: + +```bash +kuri-skill advanced # Print the path to the advanced reference +``` + +Or read the reference directly: + +```bash +read skills/pi-kuri-plugin/references/ADVANCED.md +``` + +The canonical advanced reference is at `skills/pi-kuri-advanced.md`. + +**What's available at a glance:** + +| Category | Capabilities | +|----------|-------------| +| ๐Ÿ–ฑ๏ธ Mouse | click, right-click, double-click, hover | +| โŒจ๏ธ Keyboard | type text, fill input, select option | +| ๐Ÿ“œ Scroll | up, down, left, right, pixel-amount | +| ๐Ÿง  JS Eval | arbitrary JavaScript, return values | +| ๐Ÿช Cookies | list, inspect security flags | +| ๐Ÿ”’ Security | audit headers, cookies, HTTPS, mixed content | +| ๐Ÿ“ก HAR | HTTP archive recording (start, stop, export) | +| ๐Ÿ“‘ Tabs & Sessions | multi-tab, multi-session orchestration | + +--- + +## Tips + +- **Prefer sessions** โ€” set `KURI_SESSION` env var to group tabs without + repeating `tab_id` on every call. +- **Call `page-info` before acting** โ€” confirm the right page is loaded. +- **Use `KURI_ALLOW_PRIVATE=1`** โ€” if you need to reach localhost or private + IPs during development, set this before starting the Kuri server. + +## Configuration + +| Variable | Default | Description | +|----------|---------|-------------| +| `KURI_BASE_URL` | `http://127.0.0.1:8080` | Kuri server URL | +| `KURI_SESSION` | `pi-kuri-skill` | Active session ID | +| `KURI_API_TOKEN` | (from `~/.kuri/api.token`) | API auth token | +| `KURI_OUTPUT` | `/tmp/kuri-*.png` | Screenshot output path | diff --git a/skills/pi-kuri-plugin/package.json b/skills/pi-kuri-plugin/package.json new file mode 100644 index 0000000..69218f7 --- /dev/null +++ b/skills/pi-kuri-plugin/package.json @@ -0,0 +1,23 @@ +{ + "name": "pi-kuri-skill", + "version": "1.0.0", + "description": "Pi skill for Kuri browser automation โ€” navigate, screenshot, interact, extract. Installable pi.dev plugin for the Kuri Zig-native browser automation server.", + "keywords": ["pi-package", "agent-skills", "kuri", "browser-automation"], + "type": "module", + "bin": { + "kuri-skill": "./scripts/kuri.js" + }, + "pi": { + "extensions": ["./pi-kuri.ts"], + "skills": ["."] + }, + "scripts": { + "test": "node scripts/kuri.js health", + "start": "node scripts/kuri.js" + }, + "dependencies": {}, + "license": "MIT", + "engines": { + "node": ">=18" + } +} diff --git a/skills/pi-kuri-plugin/pi-kuri.ts b/skills/pi-kuri-plugin/pi-kuri.ts new file mode 100644 index 0000000..4ec5f89 --- /dev/null +++ b/skills/pi-kuri-plugin/pi-kuri.ts @@ -0,0 +1,1300 @@ +/// +import { defineTool, type ExtensionAPI, type ExtensionCommandContext } from "@mariozechner/pi-coding-agent"; +import { Type } from "@sinclair/typebox"; +import { readFileSync } from "node:fs"; +import path from "node:path"; + +/** + * pi-kuri โ€” Kuri browser automation extension for Pi. + * + * Provides tools and commands for driving Kuri's HTTP API: + * - Browser navigation, snapshots, click, type, fill, select, scroll + * - Screenshots, page text, markdown, links extraction + * - Cookie/security audits + * - Session-based agent loops via X-Kuri-Session + * + * Configuration (set via env or pi config): + * KURI_BASE_URL default: http://127.0.0.1:8080 + * KURI_SESSION default: pi-kuri-session + */ + +const PI_KURI_EXTENSION_VERSION = "2026-05-20"; + +// โ”€โ”€ Help text โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +const COMMAND_HELP = [ + "/kuri - open the Kuri command picker (start, navigate, snap, click, audit)", + "/kuri-help - show registered Kuri commands", + `/extension version: ${PI_KURI_EXTENSION_VERSION}`, +]; + +const MENU_CHOICES = [ + "kuri-start - start the Kuri server (if not already running)", + "kuri-health - check if Kuri server is running", + "kuri-tab-new - open a new tab and navigate to a URL", + "kuri-navigate - navigate current tab to a URL", + "kuri-snap - get accessibility snapshot (interactive refs)", + "kuri-page-info - get current page URL, title, ready state", + "kuri-click - click an element by its eN ref", + "kuri-type - type text into an element", + "kuri-fill - fill an input value", + "kuri-select - select a dropdown option", + "kuri-screenshot - capture a screenshot", + "kuri-markdown - get page as markdown", + "kuri-links - extract all links from the current page", + "kuri-text - extract page text", + "kuri-evaluate - execute JavaScript in the page", + "kuri-cookies - list cookies with security flags", + "kuri-headers - check security response headers", + "kuri-audit - run full security audit", + "kuri-back - browser back", + "kuri-forward - browser forward", + "kuri-reload - reload the page", + "kuri-session - set the active Kuri session", + "kuri-base - set the Kuri server base URL", + "kuri-token [token] - show or set the Kuri API auth token", +]; + +// โ”€โ”€ Configuration helpers โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +function kuriBaseUrl(): string { + return process.env.KURI_BASE_URL || "http://127.0.0.1:8080"; +} + +function kuriSession(): string { + return process.env.KURI_SESSION || "pi-kuri-session"; +} + +function kuriAuthToken(): string | undefined { + // 1. Try explicit env vars + if (process.env.KURI_SECRET) return process.env.KURI_SECRET; + if (process.env.KURI_API_TOKEN) return process.env.KURI_API_TOKEN; + // 2. Read from api.token file (auto-generated by new Kuri builds) + try { + const homedir = process.env.HOME || process.env.HOMEPATH || ""; + return readFileSync(`${homedir}/.kuri/api.token`, "utf-8").trim(); + } catch { + return undefined; + } +} + +function kuriHeaders(): Record { + const headers: Record = { + "X-Kuri-Session": kuriSession(), + "Accept": "application/json", + }; + const token = kuriAuthToken(); + if (token) { + headers["Authorization"] = `Bearer ${token}`; + } + return headers; +} + +function parseSiteScalar(value: string): string { + const clean = value.trim().replace(/\s+#.*$/, "").trim(); + if ((clean.startsWith('"') && clean.endsWith('"')) || (clean.startsWith("'") && clean.endsWith("'"))) { + return clean.slice(1, -1); + } + return clean; +} + +function readSiteManifest(projectRoot: string): any { + const manifestPath = path.join(projectRoot, ".sandcastle", "site-manifest.yaml"); + const raw = readFileSync(manifestPath, "utf-8"); + const manifest: any = { pages: {}, auth: {}, kuri: {} }; + let section = ""; + let pageName = ""; + for (const line of raw.split(/\r?\n/)) { + if (!line.trim() || line.trim().startsWith("#")) continue; + const top = line.match(/^([A-Za-z0-9_-]+):\s*(.*)$/); + if (top) { + section = top[1]; + pageName = ""; + if (top[2]) manifest[section] = parseSiteScalar(top[2]); + else if (section === "pages") manifest.pages = manifest.pages || {}; + else if (section === "auth") manifest.auth = manifest.auth || {}; + else if (section === "kuri") manifest.kuri = manifest.kuri || {}; + continue; + } + const nested = line.match(/^\s{2}([A-Za-z0-9_-]+):\s*(.*)$/); + if (nested && section === "pages") { + pageName = nested[1]; + manifest.pages[pageName] = nested[2] ? { url: parseSiteScalar(nested[2]) } : {}; + continue; + } + if (nested && (section === "auth" || section === "kuri")) { + manifest[section][nested[1]] = parseSiteScalar(nested[2]); + continue; + } + const pageField = line.match(/^\s{4}([A-Za-z0-9_-]+):\s*(.*)$/); + if (pageField && section === "pages" && pageName) { + manifest.pages[pageName][pageField[1]] = parseSiteScalar(pageField[2]); + } + } + return manifest; +} + +async function kuriFetch(path: string, query: Record = {}): Promise { + const base = kuriBaseUrl().replace(/\/+$/, ""); + const params = new URLSearchParams(query).toString(); + const url = `${base}${path}${params ? `?${params}` : ""}`; + return fetch(url, { headers: kuriHeaders(), signal: AbortSignal.timeout(30_000) }); +} + +async function kuriFetchText(path: string, query: Record = {}): Promise { + const resp = await kuriFetch(path, query); + if (!resp.ok) { + const body = await resp.text().catch(() => ""); + throw new Error(`Kuri ${path}: ${resp.status} ${resp.statusText} โ€” ${body.slice(0, 200)}`); + } + return resp.text(); +} + +/** + * Retry wrapper for CDP-fragile endpoints (page/info, screenshot, evaluate, markdown, etc.). + * Retries up to maxRetries times with 1s exponential backoff on 502 (CDP failure). + */ +async function kuriFetchTextWithRetry(path: string, query: Record = {}, maxRetries = 2): Promise { + let lastError: Error | null = null; + for (let attempt = 0; attempt <= maxRetries; attempt++) { + try { + return await kuriFetchText(path, query); + } catch (err: any) { + const isCdpFailure = err.message?.includes("502") || err.message?.includes("CDP"); + if (isCdpFailure && attempt < maxRetries) { + lastError = err; + const delay = (attempt + 1) * 1000; + await new Promise(r => setTimeout(r, delay)); + continue; + } + throw err; + } + } + throw lastError || new Error(`Kuri ${path}: max retries exceeded`); +} + +/** + * Resolve the current session tab ID by calling /page/info. + * Returns the tab_id string or throws if unavailable. + */ +async function resolveSessionTabId(): Promise { + const infoText = await kuriFetchTextWithRetry("/page/info", {}); + const info = JSON.parse(infoText); + if (info.tab_id) return String(info.tab_id); + throw new Error("Could not resolve current session tab ID"); +} + +function singleQuotedJsString(value: string): string { + return `'${String(value).replace(/\\/g, "\\\\").replace(/'/g, "\\'").replace(/\r/g, "\\r").replace(/\n/g, "\\n")}'`; +} + +function loginPageWarning(pageInfo: any, text: string, expectedUrl: string): string { + const finalUrl = String(pageInfo?.url || ""); + const title = String(pageInfo?.title || ""); + const lowerText = String(text || "").toLowerCase(); + const compactText = lowerText.replace(/\s+/g, " ").trim(); + const hasLoginAction = /\b(log\s*in|login|sign\s*in|authenticate)\b/.test(`${title} ${compactText}`); + const urlLooksLogin = /\/login\b|login\.php|auth|authenticate/i.test(finalUrl); + const titleLooksLogin = /\b(authenticate|login|sign in)\b/i.test(title); + let expectedHash = ""; + let finalHash = ""; + try { expectedHash = new URL(expectedUrl).hash || ""; } catch { /* ignore */ } + try { finalHash = new URL(finalUrl).hash || ""; } catch { /* ignore */ } + const hashMismatch = Boolean(expectedHash && finalHash && expectedHash !== finalHash); + const littleContent = compactText.length > 0 && compactText.length < 500; + if ((urlLooksLogin || titleLooksLogin || hasLoginAction) && (littleContent || hashMismatch || urlLooksLogin || titleLooksLogin)) { + return "There is strong evidence this capture may be an unauthenticated login page: little page content and login/authentication signals were detected. The reviewer should verify the provided screenshot is for the correct page."; + } + return ""; +} + +async function checkKuriHealth(): Promise { + try { + const resp = await kuriFetch("/health"); + return resp.ok; + } catch { + return false; + } +} + +// โ”€โ”€ Tool definition helper โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +/** + * Wraps defineTool to give `execute` a typed `params` argument. + * The type is inferred from the TypeBox schema via the `S` generic. + */ +function kuriTool>( + def: Omit[0], "parameters" | "execute"> & { + parameters: S; + execute: ( + toolCallId: string, + params: Record, + signal: AbortSignal, + onUpdate: (update: unknown) => void, + ) => Promise | unknown; + }, +) { + return defineTool(def); +} + +// โ”€โ”€ Tool definitions โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +const kuriNavigateTool = kuriTool({ + name: "kuri_navigate", + label: "Navigate Browser", + description: "Navigate a Kuri browser tab to a URL and return the page info.", + promptSnippet: "Navigate browser to a URL", + promptGuidelines: [ + "Use kuri_navigate to open a web page in the Kuri-controlled browser.", + "After navigation, take a snapshot to get interactive element refs.", + ], + parameters: Type.Object({ + url: Type.String({ description: "The URL to navigate to" }), + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID. Defaults to session tab." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = { url: String(params.url) }; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText("/navigate", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriSnapTool = kuriTool({ + name: "kuri_snap", + label: "Browser Snapshot", + description: "Get an accessibility snapshot of the current page with interactive element refs (eN). Use filter=interactive&format=compact for low-token agent loops.", + promptSnippet: "Get browser accessibility snapshot", + promptGuidelines: [ + "Use kuri_snap after navigation to get clickable/typeable element references.", + "Each element has a ref like e0, e1, e2 that can be used with kuri_click, kuri_type, etc.", + "Filter to 'interactive' to get only clickable/typeable elements.", + "Use format 'compact' for minimal token usage.", + "Re-snap after any significant interaction โ€” refs are snapshot-local.", + ], + parameters: Type.Object({ + filter: Type.Optional(Type.String({ description: "Filter: 'interactive' (default) or 'all'" })), + format: Type.Optional(Type.String({ description: "Format: 'compact' (default) or 'json'" })), + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = { + filter: String(params.filter || "interactive"), + format: String(params.format || "compact"), + }; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText("/snapshot", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriClickTool = kuriTool({ + name: "kuri_click", + label: "Click Browser Element", + description: "Click an element in the browser by its accessibility snapshot ref (e.g. e0, e1).", + promptSnippet: "Click browser element by ref", + promptGuidelines: [ + "Use the ref from a kuri_snap snapshot (e.g. e0, e1, e2).", + "Re-snap after clicking to get updated state.", + ], + parameters: Type.Object({ + ref: Type.String({ description: "Element ref from snapshot (e.g. 'e0', 'e1')" }), + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = { action: "click", ref: String(params.ref) }; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText("/action", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriTypeTool = kuriTool({ + name: "kuri_type", + label: "Type in Browser", + description: "Type text into an element by its accessibility snapshot ref.", + promptSnippet: "Type text into browser element", + promptGuidelines: [ + "Use the ref from a kuri_snap snapshot.", + "Clears existing text before typing by default.", + ], + parameters: Type.Object({ + ref: Type.String({ description: "Element ref from snapshot (e.g. 'e0')" }), + value: Type.String({ description: "Text to type into the element" }), + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = { + action: "type", + ref: String(params.ref), + value: String(params.value), + }; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText("/action", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriFillTool = kuriTool({ + name: "kuri_fill", + label: "Fill Input in Browser", + description: "Fill an input element's value (sets value directly, no key events).", + promptSnippet: "Fill browser input element", + parameters: Type.Object({ + ref: Type.String({ description: "Element ref from snapshot (e.g. 'e0')" }), + value: Type.String({ description: "Value to set" }), + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = { + action: "fill", + ref: String(params.ref), + value: String(params.value), + }; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText("/action", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriSelectTool = kuriTool({ + name: "kuri_select", + label: "Select Dropdown Option", + description: "Select an option from a dropdown/select element by ref.", + promptSnippet: "Select dropdown option", + parameters: Type.Object({ + ref: Type.String({ description: "Element ref from snapshot (e.g. 'e0')" }), + value: Type.String({ description: "Option value to select" }), + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = { + action: "select", + ref: String(params.ref), + value: String(params.value), + }; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText("/action", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriPageInfoTool = kuriTool({ + name: "kuri_page_info", + label: "Page Info", + description: "Get current page URL, title, ready state, viewport, and scroll position.", + promptSnippet: "Get current browser page info", + parameters: Type.Object({ + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = {}; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchTextWithRetry("/page/info", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriScreenshotTool = kuriTool({ + name: "kuri_screenshot", + label: "Browser Screenshot", + description: "Capture a browser screenshot (base64-encoded).", + promptSnippet: "Capture browser screenshot", + promptGuidelines: [ + "Use settle_ms when capturing heavy JS-rendered pages to allow content to finish loading.", + ], + parameters: Type.Object({ + format: Type.Optional(Type.String({ description: "Image format: 'png' (default) or 'jpeg'" })), + quality: Type.Optional(Type.Integer({ description: "JPEG quality 1-100 (default: 80, only applies to jpeg)" })), + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + settle_ms: Type.Optional(Type.Number({ description: "Milliseconds to wait after navigation before capture (default: 0). Use on heavy JS pages." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = {}; + if (params.format) query.format = String(params.format); + if (params.quality !== undefined && params.quality !== null) query.quality = String(params.quality); + if (params.tab_id) query.tab_id = String(params.tab_id); + const settleMs = params.settle_ms ? Number(params.settle_ms) : 0; + if (settleMs > 0) { + await new Promise(r => setTimeout(r, settleMs)); + } + const text = await kuriFetchTextWithRetry("/screenshot", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriEvaluateTool = kuriTool({ + name: "kuri_evaluate", + label: "Execute JavaScript", + description: "Execute JavaScript in the browser page and get the result.", + promptSnippet: "Execute JavaScript in browser", + parameters: Type.Object({ + expression: Type.String({ description: "JavaScript expression to evaluate" }), + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = { expression: String(params.expression) }; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchTextWithRetry("/evaluate", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriMarkdownTool = kuriTool({ + name: "kuri_markdown", + label: "Page as Markdown", + description: "Convert the current page to Markdown.", + promptSnippet: "Convert page to markdown", + parameters: Type.Object({ + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID. Auto-resolves if omitted." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = {}; + if (params.tab_id) { + query.tab_id = String(params.tab_id); + } else { + // Auto-resolve current tab since Kuri backend requires tab_id for /markdown + query.tab_id = await resolveSessionTabId(); + } + const text = await kuriFetchTextWithRetry("/markdown", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriLinksTool = kuriTool({ + name: "kuri_links", + label: "Extract Page Links", + description: "Extract all links from the current page.", + promptSnippet: "Extract page links", + parameters: Type.Object({ + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID. Auto-resolves if omitted." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = {}; + if (params.tab_id) { + query.tab_id = String(params.tab_id); + } else { + // Auto-resolve current tab since Kuri backend requires tab_id for /links + query.tab_id = await resolveSessionTabId(); + } + const text = await kuriFetchTextWithRetry("/links", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriTextTool = kuriTool({ + name: "kuri_text", + label: "Extract Page Text", + description: "Extract plain text from the current page. Falls back to aria-labels and visible text if raw text is empty.", + promptSnippet: "Extract page text", + promptGuidelines: [ + "On JS-heavy SPAs, kuri_text may return empty string. The tool auto-falls back to aria-label extraction.", + "If the result still seems sparse, try kuri_snap to see visible interactive elements.", + ], + parameters: Type.Object({ + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = {}; + if (params.tab_id) query.tab_id = String(params.tab_id); + let rawText = await kuriFetchTextWithRetry("/text", query); + let text = rawText; + // Parse the JSON response to check if the extracted value is actually empty + try { + const parsed = JSON.parse(rawText); + const extractedValue = parsed?.result?.result?.value || parsed?.result?.value || ""; + if (extractedValue && String(extractedValue).trim()) { + text = String(extractedValue); + } else { + // Value is empty โ€” trigger fallback + text = ""; + } + } catch { + // Not JSON, use raw + } + // If extracted text is empty or whitespace-only, fall back to aria-labels + // and visible leaf text via evaluate + if (!text || !text.trim()) { + try { + const evalQuery: Record = { + expression: [ + "(function(){", + " var t=[];", + " var els=document.querySelectorAll('[aria-label],[role],a,button');", + " for(var i=0;i { + const query: Record = {}; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText("/cookies", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriAuditTool = kuriTool({ + name: "kuri_audit", + label: "Security Audit", + description: "Run a full security audit on the current page: HTTPS, security headers, cookies exposed to JS.", + promptSnippet: "Run browser security audit", + parameters: Type.Object({ + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = {}; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText("/audit", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriNewTabTool = kuriTool({ + name: "kuri_new_tab", + label: "New Browser Tab", + description: "Create a new browser tab, optionally navigating to a URL.", + promptSnippet: "Create new browser tab", + parameters: Type.Object({ + url: Type.Optional(Type.String({ description: "URL to navigate to in the new tab" })), + activate: Type.Optional(Type.String({ description: "Set as the active/session tab (default: true)" })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = {}; + if (params.url) query.url = String(params.url); + if (params.activate === "false") query.activate = "false"; + const text = await kuriFetchText("/tab/new", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriScrollTool = kuriTool({ + name: "kuri_scroll", + label: "Scroll Browser Page", + description: "Scroll the page or a specific element. Direction: up, down, left, right.", + promptSnippet: "Scroll browser page", + parameters: Type.Object({ + ref: Type.Optional(Type.String({ description: "Element ref to scroll into view (optional). Scrolls page if omitted." })), + direction: Type.Optional(Type.String({ description: "Scroll direction: 'up', 'down', 'left', 'right'" })), + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = {}; + if (params.ref) { query.action = "scrollintoview"; query.ref = String(params.ref); } + if (params.direction) query.direction = String(params.direction); + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText(params.ref ? "/scrollintoview" : "/action", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriScrollUpTool = kuriTool({ + name: "kuri_scroll_up", + label: "Scroll Page Up", + description: "Scroll the browser page up.", + promptSnippet: "Scroll page up", + parameters: Type.Object({ + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = { action: "scroll", direction: "up" }; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText("/action", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriScrollDownTool = kuriTool({ + name: "kuri_scroll_down", + label: "Scroll Page Down", + description: "Scroll the browser page down.", + promptSnippet: "Scroll page down", + parameters: Type.Object({ + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = { action: "scroll", direction: "down" }; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText("/action", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriScreenshotSiteTool = kuriTool({ + name: "kuri_screenshot_site", + label: "Screenshot Authenticated Page (Site Manifest)", + description: "One-shot full flow: use a page from .sandcastle/site-manifest.yaml or an explicit simulator URL, apply repository-defined auth, navigate, settle, and save a screenshot into Sandcastle evidence.", + promptSnippet: "One-shot authenticated screenshot for verifier/reviewer evidence lanes. Prefer page=\"\" or an explicit simulator URL before manual browser tools.", + promptGuidelines: [ + "When your role is responsible for visual evidence and this tool is available, prefer kuri_screenshot_site before trying individual kuri_* tools or gui_* tools.", + "This is the one-shot tool: it handles auth, navigation, settle timing, and PNG saving in a single call.", + "", + "Preferred: use a page name from .sandcastle/site-manifest.yaml:", + ' kuri_screenshot_site page="alert-log"', + ' kuri_screenshot_site page="dashboard"', + "", + "Also valid: use an explicit simulator URL when the ticket needs a per-worktree server:", + ' kuri_screenshot_site url="$(sandstate sim-url KEY)/indexWEB-ODAS.php#alert-log"', + "", + "Avoid output/token unless the operator explicitly asks for an override. Never write evidence to Desktop.", + "By default, the tool saves into .sandcastle/tickets//evidence/ when running inside a task worktree.", + "The tool reports URL/title and warns when the capture looks like an unauthenticated login page; reviewers must inspect that warning before accepting evidence.", + "", + 'Full-page capture: add full_page="true" to capture the entire scrollable document height.', + ' kuri_screenshot_site page="dashboard" full_page="true"', + ' kuri_screenshot_site url="http://..." full_page="true"', + "", + "Do NOT fall back to multi-step kuri_navigate + kuri_snap + kuri_screenshot unless", + "kuri_screenshot_site fails with an error you cannot fix (e.g. auth failure, wrong page).", + "Use gui_screenshot or gui_read only as a fallback when Kuri cannot reach the needed page or state.", + ], + parameters: Type.Object({ + page: Type.Optional(Type.String({ description: "Page name from site manifest (e.g. 'alert-log', 'dashboard', 'networking')" })), + url: Type.Optional(Type.String({ description: "Explicit URL for simulator/per-worktree capture. Prefer page when .sandcastle/site-manifest.yaml has a matching page." })), + project: Type.Optional(Type.String({ description: "Project root path (defaults to CWD). Used to find .sandcastle/site-manifest.yaml" })), + output: Type.Optional(Type.String({ description: "Advanced override. Prefer the default ticket evidence path; never use Desktop." })), + token: Type.Optional(Type.String({ description: "Advanced auth override. Prefer manifest auth and environment configuration." })), + settle_ms: Type.Optional(Type.Number({ description: "Milliseconds to wait after navigation before screenshot (default: from manifest or 3000)" })), + full_page: Type.Optional(Type.Boolean({ + description: "Capture the full scrollable page height (viewport resize + JPEG, default: true). Set to false to capture only the visible viewport.", +})), + tab_id: Type.Optional(Type.String({ description: "Reuse an existing tab ID instead of creating a new one" })), + }, { additionalProperties: false }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const projectRoot = params.project ? String(params.project) : process.cwd(); + const outputPath = params.output ? String(params.output) : ""; + if (outputPath && /(^|\/)Desktop(\/|$)/i.test(outputPath.replace(/\\/g, "/"))) { + return { + content: [{ + type: "text" as const, + text: "Refusing Desktop output path. Visual evidence must stay under the repository Sandcastle evidence area unless the operator explicitly asks for a different non-Desktop path.", + }], + }; + } + const authToken = params.token ? String(params.token) : undefined; + const settleMs = params.settle_ms ? Number(params.settle_ms) : undefined; + const reuseTabId = params.tab_id ? String(params.tab_id) : undefined; + const pageName = params.page ? String(params.page) : ""; + const explicitUrl = params.url ? String(params.url) : ""; + const fullPage = params.full_page !== false; + + // 1. Load site manifest or explicit URL target + let manifest: any = null; + let targetUrl = ""; + let manifestBaseUrl = "http://localhost:8082"; + let manifestSettleMs = 3000; + if (explicitUrl) { + targetUrl = explicitUrl; + try { + manifest = readSiteManifest(projectRoot); + } catch { /* explicit URL can run without a site manifest */ } + try { + const u = new URL(explicitUrl); + manifestBaseUrl = u.origin; + } catch { /* keep default */ } + } else if (pageName) { + try { + manifest = readSiteManifest(projectRoot); + manifestBaseUrl = manifest.base_url || "http://localhost:8082"; + manifestSettleMs = settleMs ?? Number(manifest.kuri?.settle_ms || 3000); + // Normalize: agent might pass underscores instead of hyphens + const pageDef = manifest.pages?.[pageName] || manifest.pages?.[pageName.replace(/_/g, "-")]; + if (!pageDef) { + const available = Object.keys(manifest.pages || {}).join(", "); + return { + content: [{ type: "text" as const, text: `Unknown page '${pageName}'. Available: ${available}` }], + }; + } + targetUrl = manifestBaseUrl.replace(/\/$/, "") + pageDef.url; + } catch (err: any) { + return { + content: [{ type: "text" as const, text: `Failed to load site manifest: ${err.message}` }], + }; + } + } else { + return { + content: [{ type: "text" as const, text: 'Must provide page="" from .sandcastle/site-manifest.yaml or url="".' }], + }; + } + + // 2. Read auth config + const authMethod = manifest?.auth?.method || "none"; + const tokenEnv = manifest?.auth?.token_env || "DEV_SCREENSHOT_TOKEN"; + const defaultToken = manifest?.auth?.default_token || ""; + const loginPath = manifest?.auth?.login_url || manifest?.auth?.login_page || ""; + const username = manifest?.auth?.username || ""; + const password = manifest?.auth?.password || ""; + const usernameSelector = manifest?.auth?.username_selector || "input[name=\"authName\"]"; + const passwordSelector = manifest?.auth?.password_selector || "input[name=\"authPassword\"]"; + const submitSelector = manifest?.auth?.submit_selector || "button[type=\"submit\"]"; + const resolvedToken = authToken || process.env[tokenEnv] || defaultToken; + + // 3. Ensure Kuri is healthy + const healthResp = await kuriFetch("/health"); + if (!healthResp.ok) { + return { + content: [{ type: "text" as const, text: `Kuri server not responding (${healthResp.status}) โ€” start it first with 'kuri-start' or 'KURI_ALLOW_PRIVATE=1 kuri'` }], + }; + } + + // 4. Get or create tab + let tabId = reuseTabId || ""; + let newTabResp = ""; + if (!tabId) { + newTabResp = await kuriFetchText("/tab/new", { url: "about:blank", activate: "true" }); + try { + const tabData = JSON.parse(newTabResp); + tabId = tabData.tab_id || tabData.id || ""; + } catch { /* fall through */ } + } + if (!tabId) { + return { + content: [{ type: "text" as const, text: `Failed to create Kuri tab. Kuri may not be running. Response: ${newTabResp.slice(0, 200)}` }], + }; + } + + // 5. Auth is repository-owned manifest behavior, not pi-dev product knowledge. + if (authMethod === "form-login") { + const originUrl = manifestBaseUrl.replace(/\/$/, ""); + const loginPageUrl = loginPath.startsWith("http") ? loginPath : originUrl + (loginPath || "/login.php"); + + try { + await kuriFetchText("/navigate", { tab_id: tabId, url: loginPageUrl }); + await new Promise(r => setTimeout(r, 1500)); + + await kuriFetchText("/evaluate", { + tab_id: tabId, + expression: `document.querySelector(${singleQuotedJsString(usernameSelector)}).value=${JSON.stringify(username)}`, + }); + await kuriFetchText("/evaluate", { + tab_id: tabId, + expression: `document.querySelector(${singleQuotedJsString(passwordSelector)}).value=${JSON.stringify(password)}`, + }); + await new Promise(r => setTimeout(r, 300)); + + await kuriFetchText("/evaluate", { + tab_id: tabId, + expression: `document.querySelector(${singleQuotedJsString(submitSelector)}).click()`, + }); + + await new Promise(r => setTimeout(r, 3000)); + } catch (err: any) { + return { + content: [{ type: "text" as const, text: `Login failed (${err.message}). The captured screenshot may show an unauthenticated page.` }], + }; + } + } else if (authMethod === "bearer-token-session" && resolvedToken && loginPath) { + const originUrl = manifestBaseUrl.replace(/\/$/, ""); + const loginPageUrl = loginPath.startsWith("http") ? loginPath : originUrl + loginPath; + await kuriFetchText("/navigate", { + tab_id: tabId, + url: loginPageUrl, + headers: JSON.stringify({ Authorization: `Bearer ${resolvedToken}` }), + }); + await new Promise(r => setTimeout(r, 1500)); + } + + // 6. Navigate to target + await kuriFetchText("/navigate", { + tab_id: tabId, + url: targetUrl, + }); + + // 7. Settle + const settle = settleMs ?? manifestSettleMs; + await new Promise(r => setTimeout(r, settle)); + + // 7.5 Full-page mode: resize viewport to full page height so the screenshot + // captures the entire scrollable document instead of just the viewport. + // Uses JPEG format for tall pages since Chrome's PNG encoder can OOM. + // Resets viewport after capture. + let originalViewport: { w: number; h: number } | null = null; + let useJpegForTall = false; + if (fullPage) { + try { + const piText = await kuriFetchText("/page/info", { tab_id: tabId }).catch(() => "{}"); + const pi: any = JSON.parse(piText); + const vw = Number(pi.viewport_width) || 1920; + const vh = Number(pi.viewport_height) || 1080; + originalViewport = { w: vw, h: vh }; + const dimText = await kuriFetchText("/evaluate", { + tab_id: tabId, + expression: "JSON.stringify({sw:document.documentElement.scrollWidth,sh:document.documentElement.scrollHeight})", + }).catch(() => ""); + const dimJson: any = JSON.parse(dimText); + const sw = Number(dimJson?.result?.result?.value ? JSON.parse(dimJson.result.result.value).sw : 0) || vw; + const sh = Number(dimJson?.result?.result?.value ? JSON.parse(dimJson.result.result.value).sh : 0) || vh; + if (sh > vh && sh > 0 && sw > 0) { + useJpegForTall = true; + await kuriFetch("/set/viewport", { tab_id: tabId, width: String(sw), height: String(sh) }); + // Let the renderer catch up after resize + await new Promise(r => setTimeout(r, 1000)); + } + } catch { + // Best-effort: if viewport resize fails, fall through to normal screenshot + } + } + + // 8. Capture advisory page state before screenshot so the reviewer can tell + // whether auth landed on the expected page or a login wall. + const pageInfoText = await kuriFetchText("/page/info", { tab_id: tabId }).catch(() => ""); + let pageInfo: any = {}; + try { + pageInfo = pageInfoText ? JSON.parse(pageInfoText) : {}; + } catch { /* advisory only */ } + const pageText = await kuriFetchText("/text", { tab_id: tabId }).catch(() => ""); + const warning = loginPageWarning(pageInfo, pageText, targetUrl); + + // 9. Screenshot โ€” use JPEG for tall pages (full-page mode) since + // Chrome's PNG encoder may OOM on very tall viewport captures + const screenshotResp = await kuriFetch("/screenshot", { + tab_id: tabId, + ...(useJpegForTall ? { format: "jpeg", quality: "30" } : {}), + }); + const screenshotJson = await screenshotResp.json(); + let pngData: string | undefined; + if (screenshotJson.result?.result?.data) { + pngData = screenshotJson.result.result.data; + } else if (screenshotJson.result?.data) { + pngData = screenshotJson.result.data; + } else if (screenshotJson.data) { + pngData = screenshotJson.data; + } + + if (!pngData) { + return { + content: [{ type: "text" as const, text: `Screenshot failed: Kuri returned an unexpected response format. The page may not have loaded correctly. ${JSON.stringify(screenshotJson).slice(0, 200)}` }], + }; + } + + // 10. Save PNG โ€” default to ticket-owned Sandcastle evidence, with explicit + // non-Desktop override allowed. + const pageSlug = pageName || targetUrl.replace(/[^a-z0-9]/gi, "-").replace(/-+/g, "-").slice(0, 40).toLowerCase(); + let savedPath = outputPath; + let repoRoot = ""; + let taskKey = ""; + if (!savedPath) { + const wtMatch = projectRoot.match(/(.+?)\/\.sandcastle\/worktrees\/sandcastle-([A-Z0-9]{2,5})-/i); + if (wtMatch) { + repoRoot = wtMatch[1]; + taskKey = wtMatch[2].toLowerCase(); + const stamp = new Date().toISOString().replace(/[:.]/g, "-"); + savedPath = path.join(repoRoot, ".sandcastle", "tickets", taskKey, "evidence", `kuri-${pageSlug}-${stamp}`, "visual-evidence.png"); + } else { + savedPath = path.join(projectRoot, ".sandcastle", "evidence", "manual", `evidence-${pageSlug}-${Date.now()}.png`); + } + } + const buf = Buffer.from(pngData, "base64"); + const { mkdirSync, writeFileSync } = require("node:fs") as typeof import("node:fs"); + mkdirSync(path.dirname(savedPath), { recursive: true }); + writeFileSync(savedPath, buf); + + // 10.5 Full-page mode: reset viewport to original dimensions + if (originalViewport) { + kuriFetch("/set/viewport", { + tab_id: tabId, + width: String(originalViewport.w), + height: String(originalViewport.h), + }).catch(() => {}); + } + + // Warn if screenshot is suspiciously small (blank/error page) + if (buf.length < 2000) { + return { + content: [{ type: "text" as const, text: `Evidence file ${savedPath} saved (only ${buf.length} bytes โ€” page may be blank or returned an error). Inspect before relying on this evidence.` }], + }; + } + + let pageTitle = ""; + let pageUrl = ""; + pageTitle = pageInfo.title || ""; + pageUrl = pageInfo.url || ""; + const artifactPath = repoRoot && savedPath.startsWith(repoRoot) + ? path.relative(repoRoot, savedPath) + : savedPath; + const evidenceManifest = { + status: "passed", + role: "reviewer", + screen: targetUrl, + finalUrl: pageUrl, + title: pageTitle, + page: pageName, + capturedAt: new Date().toISOString(), + artifacts: [artifactPath], + checks: [ + { name: "visual evidence", passed: true, detail: "Kuri screenshot captured" }, + { name: "page rendered", passed: true, detail: "document body exists" }, + ], + warnings: warning ? [warning] : [], + ...(taskKey ? { taskKey: taskKey.toUpperCase() } : {}), + }; + const manifestJson = `${JSON.stringify(evidenceManifest, null, 2)}\n`; + writeFileSync(path.join(path.dirname(savedPath), "visual-evidence.json"), manifestJson); + writeFileSync(path.join(path.dirname(savedPath), "visual-evidence-manifest.json"), manifestJson); + + return { + content: [{ + type: "text" as const, + text: [ + `Evidence file ${savedPath} saved (${buf.length} bytes)`, + `Manifest ${path.join(path.dirname(savedPath), "visual-evidence-manifest.json")} saved`, + outputPath ? "Note: output override was used; prefer the default Sandcastle evidence path for ticket handoff." : "", + authToken ? "Note: token override was used; prefer repository manifest auth/env config." : "", + warning ? `Warning: ${warning}` : "", + `Page: ${pageTitle}`, + `URL: ${pageUrl}`, + ].filter(Boolean).join("\n"), + }], + }; + }, +}); + +const kuriConsoleErrorsTool = kuriTool({ + name: "kuri_console_errors", + label: "Console Errors", + description: "Extract recent console.error/warn entries from the browser page. Useful for diagnosing why a page isn't rendering.", + promptSnippet: "Get browser console errors", + promptGuidelines: [ + "Use after a page loads but seems empty or broken โ€” checks for JS errors.", + "If the result is empty, no errors were captured (or the console was cleared).", + ], + parameters: Type.Object({ + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const expression = [ + "(function(){", + " var e = window.__kuriErrorLog || [];", + " if (!window.__kuriErrorLog) {", + " window.__kuriErrorLog = [];", + " var orig = console.error;", + " console.error = function() {", + " var args = Array.prototype.slice.call(arguments);", + " window.__kuriErrorLog.push(args.map(function(a){return typeof a === 'string' ? a : JSON.stringify(a)}).join(' '));", + " return orig.apply(console, arguments);", + " };", + " var origWarn = console.warn;", + " console.warn = function() {", + " var args = Array.prototype.slice.call(arguments);", + " window.__kuriErrorLog.push('[WARN] ' + args.map(function(a){return typeof a === 'string' ? a : JSON.stringify(a)}).join(' '));", + " return origWarn.apply(console, arguments);", + " };", + " }", + " return e.length ? e.slice(-30).join('\\n') : '';", + " })()", + ].join(""); + const query: Record = { expression }; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchTextWithRetry("/evaluate", query); + let result = ""; + try { + const parsed = JSON.parse(text); + result = parsed?.result?.result?.value || parsed?.result?.value || ""; + } catch { + result = text; + } + if (!result || !String(result).trim()) { + result = "No console errors captured (console.error interceptor installed)."; + } + return { content: [{ type: "text" as const, text: String(result) }] }; + }, +}); + +const allTools = [ + kuriNavigateTool, + kuriSnapTool, + kuriClickTool, + kuriTypeTool, + kuriFillTool, + kuriSelectTool, + kuriPageInfoTool, + kuriScreenshotTool, + kuriEvaluateTool, + kuriMarkdownTool, + kuriLinksTool, + kuriTextTool, + kuriConsoleErrorsTool, + kuriCookiesTool, + kuriAuditTool, + kuriNewTabTool, + kuriScrollTool, + kuriScrollUpTool, + kuriScrollDownTool, + kuriScreenshotSiteTool, +]; + +// โ”€โ”€ Slash command handlers โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +function isDuplicateToolConflict(error: unknown): boolean { + return error instanceof Error && !!error.message?.includes("already defined"); +} + +async function cmdKuriStart(_args: string, ctx: ExtensionCommandContext): Promise { + const ok = await checkKuriHealth(); + if (ok) { + ctx.ui.notify("Kuri server is already running."); + return; + } + ctx.ui.notify("Kuri is not running. Start it manually:\n kuri\n\nOr with options:\n PORT=8080 kuri"); +} + +async function cmdKuriHealth(_args: string, ctx: ExtensionCommandContext): Promise { + const ok = await checkKuriHealth(); + if (ok) { + const resp = await kuriFetch("/health"); + const data = await resp.text(); + ctx.ui.notify(`Kuri server is running.\n${data.slice(0, 500)}`); + } else { + ctx.ui.notify("Kuri server is NOT running. Start it with: kuri"); + } +} + +function cmdKuriTabNew(args: string, ctx: ExtensionCommandContext): void { + ctx.ui.notify(`Use the kuri_new_tab tool with url="${args.trim() || ''}"`); +} + +function cmdKuriNavigate(args: string, ctx: ExtensionCommandContext): void { + ctx.ui.notify(`Use the kuri_navigate tool with url="${args.trim() || ''}"`); +} + +function cmdKuriSnap(_args: string, ctx: ExtensionCommandContext): void { + ctx.ui.notify("Use the kuri_snap tool with filter=interactive&format=compact"); +} + +function cmdBase(args: string, ctx: ExtensionCommandContext): void { + const url = args.trim(); + if (url) { + process.env.KURI_BASE_URL = url; + ctx.ui.notify(`KURI_BASE_URL set to ${url}`); + } else { + ctx.ui.notify(`Current KURI_BASE_URL: ${kuriBaseUrl()}`); + } +} + +function cmdSession(args: string, ctx: ExtensionCommandContext): void { + const id = args.trim(); + if (id) { + process.env.KURI_SESSION = id; + ctx.ui.notify(`KURI_SESSION set to ${id}`); + } else { + ctx.ui.notify(`Current KURI_SESSION: ${kuriSession()}`); + } +} + +function cmdToken(args: string, ctx: ExtensionCommandContext): void { + const token = args.trim(); + if (token) { + process.env.KURI_SECRET = token; + ctx.ui.notify("KURI_SECRET (API auth token) set."); + } else { + const current = kuriAuthToken(); + if (current) { + ctx.ui.notify(`Auth token: ${current.slice(0, 20)}...`); + } else { + ctx.ui.notify("No auth token found. Set via KURI_SECRET env var or ~/.kuri/api.token file."); + } + } +} + +async function cmdGenericKuri(action: string, _args: string, ctx: ExtensionCommandContext): Promise { + const ok = await checkKuriHealth(); + if (!ok) { + ctx.ui.notify("Kuri server is not running. Start it first with 'kuri'."); + return; + } + ctx.ui.notify(`Run: kuri_${action} tool`); +} + +// โ”€โ”€ Extension entry point โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +export default function kuriCommands(pi: ExtensionAPI): void { + for (const tool of allTools) { + try { + pi.registerTool(tool); + } catch (error) { + if (!isDuplicateToolConflict(error)) { + throw error; + } + } + } + + // Register /kuri-photo slash command for easy one-shot screenshots + pi.registerCommand("kuri-photo", { + description: "One-shot site-manifest screenshot. Usage: /kuri-photo [project-root]", + handler: async (args, ctx) => { + const parts = args.trim().split(/\s+/); + const pageName = parts[0] || ""; + const projectRoot = parts[1] || process.cwd(); + ctx.ui.notify(`Use the kuri_screenshot_site tool with page="${pageName}" project="${projectRoot}"`); + }, + }); + + pi.on("session_start", async (_event, ctx) => { + ctx.ui.setStatus("kuri", `๐ŸŒฐ Kuri ${PI_KURI_EXTENSION_VERSION}`); + }); + + pi.on("session_shutdown", async (_event, ctx) => { + ctx.ui.setStatus("kuri", undefined); + }); + + pi.on("before_agent_start", async (event) => { + const baseUrl = kuriBaseUrl(); + const sessionId = kuriSession(); + const kuriGuide = [ + "", + `## Kuri Browser Automation (extension ${PI_KURI_EXTENSION_VERSION})`, + "", + "Kuri is a Zig-native browser automation server. Use it only when your role's tool list includes Kuri tools.", + "If your role does not include `kuri_screenshot_site`, do not attempt browser capture; record the target page, selector, or URL for the verifier/reviewer evidence lane.", + "", + "### 1. One-Shot Screenshot (PREFERRED)", + "", + 'Verifier/reviewer lanes should use `kuri_screenshot_site` for visual evidence. This handles auth, navigation,', + "settle timing, and evidence PNG saving in a single call:", + "", + ' kuri_screenshot_site page="alert-log" # from .sandcastle/site-manifest.yaml', + ' kuri_screenshot_site page="dashboard"', + "", + "For per-worktree simulators, an explicit URL is also valid:", + ' kuri_screenshot_site url="$(sandstate sim-url KEY)/indexWEB-ODAS.php#networking"', + "", + "Avoid output/token unless the operator explicitly asks for an override.", + "Never write evidence to Desktop; default output stays under .sandcastle evidence paths.", + "Read the tool response after capture. If it warns that the page looks like an unauthenticated login page, verify or retry before treating the screenshot as proof.", + "", + "When Kuri tools are available, use this first before manual browser tools or gui_* tools.", + "", + "### 2. Manual Browser Tools (fallback only if screenshot_site fails)", + "", + "Only use these when kuri_screenshot_site cannot reach the right page or state.", + "", + "**Open a tab:**", + " kuri_new_tab(url?)", + "", + "**Check what's on the page:**", + " kuri_page_info โ€” URL, title, ready state", + " kuri_snap โ€” interactive element refs (e0, e1...)", + " kuri_screenshot โ€” capture screenshot to inspect", + "", + "**Act on elements (by ref from snap):**", + " kuri_click(ref) โ€” click element", + " kuri_type(ref, value) โ€” type text", + " kuri_fill(ref, value) โ€” fill input value", + " kuri_select(ref, value) โ€” select dropdown option", + " kuri_scroll(direction?) โ€” scroll page", + "", + "**Get page content:**", + " kuri_markdown โ€” page as markdown (auto-resolves tab)", + " kuri_links โ€” all links (auto-resolves tab)", + " kuri_text โ€” plain text (falls back to aria-labels on SPAs)", + " kuri_evaluate(expression) โ€” run JavaScript", + " kuri_console_errors โ€” diagnose JS errors when page is empty/broken", + "", + "**Tips:**", + " - CDP-fragile endpoints auto-retry (up to 3 attempts) on transient failures.", + " - Heavy JS pages: add settle_ms=5000 to kuri_screenshot for settle time.", + " - Re-snap after navigation or DOM changes โ€” refs are snapshot-local.", + "", + ].join("\n"); + return { + systemPrompt: `${event.systemPrompt}\n${kuriGuide}`, + }; + }); + + pi.registerCommand("kuri", { + description: "Open the Kuri command picker", + handler: async (_args, ctx) => { + const choice = await ctx.ui.select("Kuri commands", MENU_CHOICES); + if (!choice) return; + const prefix = choice.split(" - ")[0]; + ctx.ui.setStatus("kuri", `selected: ${prefix}`); + }, + }); + + pi.registerCommand("kuri-help", { + description: "Show registered Kuri commands", + handler: async (_args, ctx) => { + ctx.ui.notify(COMMAND_HELP.join("\n")); + }, + }); + + pi.registerCommand("kuri-start", { + description: "Check or start the Kuri server", + handler: cmdKuriStart, + }); + + pi.registerCommand("kuri-health", { + description: "Check Kuri server health", + handler: cmdKuriHealth, + }); + + pi.registerCommand("kuri-tab-new", { + description: "Open a new browser tab and optionally navigate to a URL", + handler: cmdKuriTabNew, + }); + + pi.registerCommand("kuri-navigate", { + description: "Navigate the current tab to a URL", + handler: cmdKuriNavigate, + }); + + pi.registerCommand("kuri-snap", { + description: "Get accessibility snapshot with interactive refs", + handler: cmdKuriSnap, + }); + + pi.registerCommand("kuri-session", { + description: "Set the active Kuri session ID", + handler: cmdSession, + }); + + pi.registerCommand("kuri-base", { + description: "Set the Kuri server base URL", + handler: cmdBase, + }); + + pi.registerCommand("kuri-token", { + description: "Show or set the Kuri API auth token", + handler: cmdToken, + }); + + // Register convenience forwarders for common commands + for (const action of ["page-info", "click", "type", "fill", "select", "screenshot", + "markdown", "links", "text", "evaluate", "cookies", "audit", + "back", "forward", "reload"]) { + const cmdName = `kuri-${action}`; + const handler = async (_args: string, ctx: ExtensionCommandContext) => { + await cmdGenericKuri(action.replace("-", "_"), _args, ctx); + }; + pi.registerCommand(cmdName, { + description: `Browser: ${action}`, + handler, + }); + } +} diff --git a/skills/pi-kuri-plugin/references/ADVANCED.md b/skills/pi-kuri-plugin/references/ADVANCED.md new file mode 120000 index 0000000..fd286b7 --- /dev/null +++ b/skills/pi-kuri-plugin/references/ADVANCED.md @@ -0,0 +1 @@ +../../pi-kuri-advanced.md \ No newline at end of file diff --git a/skills/pi-kuri-plugin/scripts/kuri.js b/skills/pi-kuri-plugin/scripts/kuri.js new file mode 100755 index 0000000..4c1233a --- /dev/null +++ b/skills/pi-kuri-plugin/scripts/kuri.js @@ -0,0 +1,622 @@ +#!/usr/bin/env node +/** + * kuri-skill โ€” Pi skill CLI for Kuri browser automation. + * + * Available as a standalone command after npm install -g: + * kuri-skill health + * kuri-skill navigate + * ... + * + * Or run directly: + * node scripts/kuri.js health + * + * Core operations are always available directly. + * Advanced operations (click, type, fill, audit, etc.) are documented + * in references/ADVANCED.md and invoked via 'action '. + * + * Usage: + * kuri-skill health + * kuri-skill tabs + * kuri-skill navigate [tab_id] + * kuri-skill tab-new [url] + * kuri-skill screenshot [tab_id] [--output path] + * kuri-skill page-info [tab_id] + * kuri-skill text [tab_id] + * kuri-skill markdown [tab_id] + * kuri-skill links [tab_id] + * kuri-skill snap [tab_id] + * kuri-skill action [args...] + * kuri-skill advanced # Print reference to ADVANCED.md + */ + +import { readFileSync, writeFileSync, existsSync } from "node:fs"; +import { homedir } from "node:os"; +import { resolve, dirname } from "node:path"; +import { fileURLToPath } from "node:url"; +import { execSync } from "node:child_process"; + +// โ”€โ”€ Configuration โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +const __dirname = dirname(fileURLToPath(import.meta.url)); +const SKILL_ROOT = resolve(__dirname, ".."); + +function kuriBaseUrl() { + return process.env.KURI_BASE_URL || "http://127.0.0.1:8080"; +} + +function kuriApiToken() { + if (process.env.KURI_API_TOKEN) return process.env.KURI_API_TOKEN; + if (process.env.KURI_SECRET) return process.env.KURI_SECRET; + try { + return readFileSync(`${homedir()}/.kuri/api.token`, "utf-8").trim(); + } catch { + return ""; + } +} + +function defaultSession() { + return process.env.KURI_SESSION || "pi-kuri-skill"; +} + +function sessionTabId() { + return process.env.KURI_TAB_ID || ""; +} + +function headers(session) { + const h = { + "X-Kuri-Session": session || defaultSession(), + Accept: "application/json", + }; + const token = kuriApiToken(); + if (token) h["Authorization"] = `Bearer ${token}`; + return h; +} + +// โ”€โ”€ Error recovery โ€” detect if Kuri server is not running โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +function isKuriInstalled() { + try { + execSync("command -v kuri", { stdio: "ignore" }); + return true; + } catch { + return false; + } +} + +function isBuiltLocally() { + return existsSync(resolve(SKILL_ROOT, "..", "..", "zig-out", "bin", "kuri")); +} + +function platform() { + const os = process.platform; + if (os === "darwin") return "macos"; + if (os === "linux") return "linux"; + return os; +} + +/** Print structured, LLM-parsable recovery hints when Kuri server is unreachable. */ +function printKuriNotRunning(originalError) { + const hints = { + status: "kuri_not_running", + message: "Kuri server is not reachable. The server must be running on " + + `${kuriBaseUrl()} before any browser operations work.`, + error: originalError, + platform: platform(), + kuri_on_path: isKuriInstalled(), + built_in_repo: isBuiltLocally(), + suggestions: [] + }; + + if (isKuriInstalled()) { + hints.suggestions.push({ + action: "start_kuri", + command: "kuri", + detail: "Kuri is installed on PATH. Start it with the 'kuri' command." + }); + } + + if (isBuiltLocally()) { + hints.suggestions.push({ + action: "start_kuri_local", + command: "./zig-out/bin/kuri", + cwd: resolve(SKILL_ROOT, "..", ".."), + detail: "A local build exists in the kuri repo. Start it from the repo root." + }); + } + + if (!isKuriInstalled() && !isBuiltLocally()) { + const installScript = + "curl -fsSL https://raw.githubusercontent.com/justrach/kuri/main/install.sh | sh"; + hints.suggestions.push({ + action: "install_kuri", + install_url: "https://github.com/justrach/kuri", + install_script: installScript, + detail: "Kuri is not installed. Use the one-line install script, or build from source." + }); + hints.suggestions.push({ + action: "build_from_source", + clone_url: "https://github.com/justrach/kuri.git", + build_commands: ["git clone https://github.com/justrach/kuri.git", "cd kuri", "zig build", "./zig-out/bin/kuri"], + requires_zig: true, + detail: "Clone the kuri repo and build with Zig 0.16." + }); + } + + hints.suggestions.push({ + action: "check_port_or_url", + detail: `Verify the server URL. The default is http://127.0.0.1:8080. ` + + `Override with KURI_BASE_URL env var.` + }); + + console.log(JSON.stringify(hints, null, 2)); +} + +// โ”€โ”€ HTTP helpers โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +async function kuriFetch(path, query = {}, method = "GET", body = null) { + const base = kuriBaseUrl().replace(/\/+$/, ""); + const params = new URLSearchParams(query).toString(); + const url = `${base}${path}${params ? `?${params}` : ""}`; + const opts = { + method, + headers: headers(query.session || ""), + signal: AbortSignal.timeout(30_000), + }; + if (body) { + opts.headers["Content-Type"] = "application/json"; + opts.body = JSON.stringify(body); + } + try { + const resp = await fetch(url, opts); + const ct = resp.headers.get("content-type") || ""; + if (ct.includes("image/png") || ct.includes("image/")) { + const buf = await resp.arrayBuffer(); + return { _binary: true, data: Buffer.from(buf), contentType: ct }; + } + if (!resp.ok) { + const text = await resp.text().catch(() => ""); + throw new Error(`Kuri ${path}: ${resp.status} ${resp.statusText} โ€” ${text.slice(0, 300)}`); + } + return resp.json(); + } catch (err) { + // Distinguish connection refused (server not running) from other errors + if (err?.cause?.code === "ECONNREFUSED" || err?.message?.includes("ECONNREFUSED") || err?.message?.includes("fetch failed")) { + const nfe = new Error("KURI_NOT_RUNNING"); + nfe.code = "KURI_NOT_RUNNING"; + nfe.originalMessage = err.message; + throw nfe; + } + throw err; + } +} + +// โ”€โ”€ Output helpers โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +function printJson(data) { + console.log(JSON.stringify(data, null, 2)); +} + +function printText(label, data) { + if (typeof data === "string") { + console.log(data); + return; + } + if (data._binary) { + console.log(`[binary: ${data.data.length} bytes, ${data.contentType}]`); + return; + } + // Navigate through common response shapes + let result = data; + if (data && typeof data === "object") { + if (data.result && typeof data.result === "object") { + const r = data.result; + if (r.result && r.result.value !== undefined) result = r.result.value; + else if (r.value !== undefined) result = r.value; + else if (r.text !== undefined) result = r.text; + else if (r.data !== undefined) result = r.data; + else if (r.markdown !== undefined) result = r.markdown; + else result = r; + } else if (data.value !== undefined) { + result = data.value; + } else if (data.text !== undefined) { + result = data.text; + } else if (data.markdown !== undefined) { + result = data.markdown; + } + } + if (typeof result === "string") { + console.log(result); + } else { + console.log(JSON.stringify(result, null, 2)); + } +} + +// โ”€โ”€ Core operations โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +async function cmdHealth() { + const data = await kuriFetch("/health"); + printJson(data); +} + +async function cmdTabs() { + const data = await kuriFetch("/tabs"); + printJson(data); +} + +async function cmdTabNew(url) { + const params = {}; + if (url) params.url = url; + const data = await kuriFetch("/tab/new", params); + const tabId = data?.result?.tabId || data?.result?.id || data?.id || ""; + if (tabId) { + // Store the tab ID for convenience + console.log(`Tab created: ${tabId}`); + // Also print full response + printJson(data); + } else { + printJson(data); + } +} + +async function cmdNavigate(url, tabId) { + if (!url) { + console.error("Usage: kuri.js navigate [tab_id]"); + process.exit(1); + } + if (!tabId) { + // Try to find the current tab + try { + const tabs = await kuriFetch("/tabs"); + const list = Array.isArray(tabs) ? tabs : (tabs.tabs || tabs.result || []); + const current = list.find((t) => t.current); + if (current) tabId = current.id; + } catch { /* ignore */ } + } + if (!tabId) { + console.error("No tab available. Open one first: kuri.js tab-new "); + process.exit(1); + } + const params = { url, tab_id: tabId }; + const data = await kuriFetch("/navigate", params); + printText("Navigated", data); +} + +async function resolveTabId(requested) { + if (requested) return requested; + if (sessionTabId()) return sessionTabId(); + try { + const tabs = await kuriFetch("/tabs"); + const list = Array.isArray(tabs) ? tabs : (tabs.tabs || tabs.result || []); + const current = list.find((t) => t.current); + if (current) return current.id; + } catch { /* ignore */ } + return ""; +} + +async function cmdPageInfo(tabId) { + const params = {}; + if (tabId) params.tab_id = tabId; + const data = await kuriFetch("/page/info", params); + printJson(data); + } + // Show current tab info + const current = list.find((t) => t.current); + if (current) { + console.log(JSON.stringify({ url: current.url, title: current.title, tab_id: current.id }, null, 2)); + } else { + printJson(tabs); + } + } +} + +async function cmdScreenshot(tabId, outputPath) { + tabId = await resolveTabId(tabId); + const params = {}; + if (tabId) params.tab_id = tabId; + const data = await kuriFetch("/screenshot", params); + + if (data._binary) { + const path = outputPath || `/tmp/kuri-screenshot-${Date.now()}.png`; + writeFileSync(path, data.data); + console.log(`Screenshot saved: ${path} (${data.data.length} bytes)`); + return; + } + + // Handle JSON-wrapped base64 + let raw = data; + if (data.result?.data) raw = data.result.data; + else if (data.data) raw = data.data; + else if (typeof data === "string") raw = data; + + if (typeof raw === "string") { + // Strip data URI prefix + if (raw.includes(",")) raw = raw.split(",")[1]; + const buf = Buffer.from(raw, "base64"); + const path = outputPath || `/tmp/kuri-screenshot-${Date.now()}.png`; + writeFileSync(path, buf); + console.log(`Screenshot saved: ${path} (${buf.length} bytes)`); + } else { + printJson(data); + } +} + +async function cmdText(tabId) { + tabId = await resolveTabId(tabId); + const params = {}; + if (tabId) params.tab_id = tabId; + const data = await kuriFetch("/text", params); + printText("Text", data); +} + +async function cmdMarkdown(tabId) { + tabId = await resolveTabId(tabId); + const params = {}; + if (tabId) params.tab_id = tabId; + const data = await kuriFetch("/markdown", params); + printText("Markdown", data); +} + +async function cmdLinks(tabId) { + tabId = await resolveTabId(tabId); + const params = {}; + if (tabId) params.tab_id = tabId; + const data = await kuriFetch("/links", params); + printJson(data); +} + +async function cmdSnap(tabId) { + tabId = await resolveTabId(tabId); + const params = {}; + if (tabId) params.tab_id = tabId; + try { + const data = await kuriFetch("/snapshot", { ...params, filter: "interactive", format: "compact" }); + printText("Accessibility snapshot", data); + } catch { + // Fallback to full snapshot + const data = await kuriFetch("/snapshot", params); + printText("Accessibility snapshot", data); + } +} + +// โ”€โ”€ Advanced operation dispatcher โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +async function cmdAction(action, args) { + const tabId = args.tab_id || sessionTabId(); + const params = { ...args }; + if (tabId) params.tab_id = tabId; + + switch (action) { + // Navigation + case "back": { + const data = await kuriFetch("/back", params); + printText("Back", data); + break; + } + case "forward": { + const data = await kuriFetch("/forward", params); + printText("Forward", data); + break; + } + case "reload": { + const data = await kuriFetch("/reload", params); + printText("Reload", data); + break; + } + + // Interaction + case "click": { + if (!args.ref) { console.error("Usage: click [tab_id]"); process.exit(1); } + const data = await kuriFetch("/action", { ...params, action: "click", ref: args.ref }); + printText("Click", data); + break; + } + case "type": { + if (!args.ref || !args.value) { console.error("Usage: type [tab_id]"); process.exit(1); } + const data = await kuriFetch("/action", { ...params, action: "type", ref: args.ref, value: args.value }); + printText("Type", data); + break; + } + case "fill": { + if (!args.ref || !args.value) { console.error("Usage: fill [tab_id]"); process.exit(1); } + const data = await kuriFetch("/action", { ...params, action: "fill", ref: args.ref, value: args.value }); + printText("Fill", data); + break; + } + case "select": { + if (!args.ref || !args.value) { console.error("Usage: select [tab_id]"); process.exit(1); } + const data = await kuriFetch("/action", { ...params, action: "select", ref: args.ref, value: args.value }); + printText("Select", data); + break; + } + case "scroll": { + const dir = args.direction || "down"; + const amount = args.amount || ""; + const data = await kuriFetch("/action", { ...params, action: "scroll", direction: dir, ...(amount ? { amount } : {}) }); + printText("Scroll", data); + break; + } + + // Content + case "evaluate": { + if (!args.expression) { console.error("Usage: evaluate [tab_id]"); process.exit(1); } + const data = await kuriFetch("/evaluate", { ...params, expression: args.expression }); + printText("Evaluate", data); + break; + } + + // Cookies & Security + case "cookies": { + const data = await kuriFetch("/cookies", params); + printJson(data); + break; + } + case "audit": { + const data = await kuriFetch("/audit", params); + printJson(data); + break; + } + + // Tab management + case "close-tab": { + if (!args.tab_id && !tabId) { console.error("Usage: close-tab "); process.exit(1); } + const data = await kuriFetch("/tab/close", { tab_id: args.tab_id || tabId }); + printText("Close tab", data); + break; + } + + // HAR recording + case "har": { + const data = await kuriFetch("/har", params); + printJson(data); + break; + } + case "har-start": { + const data = await kuriFetch("/har/start", params); + printJson(data); + break; + } + case "har-stop": { + const data = await kuriFetch("/har/stop", params); + printJson(data); + break; + } + + default: + console.error(`Unknown action: ${action}`); + console.error("See references/ADVANCED.md for all available actions."); + process.exit(1); + } +} + +function cmdAdvanced() { + const advPath = resolve(SKILL_ROOT, "references", "ADVANCED.md"); + console.log(`\n ๐Ÿ“– Full Kuri API reference: ${advPath}`); + console.log(" Read that file with the 'read' tool for all ~100 advanced options.\n"); +} + +// โ”€โ”€ CLI routing โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +async function main() { + const cmd = process.argv[2]; + const args = process.argv.slice(3); + + if (!cmd || cmd === "--help" || cmd === "-h") { + console.log(` +Kuri Browser Automation โ€” Pi Skill CLI + +Usage: + kuri-skill [args...] + +Core Commands: + health Check if Kuri server is running + tabs List all browser tabs + tab-new [url] Open a new tab (optionally navigate to URL) + navigate [tab_id] Navigate tab to URL + page-info [tab_id] Get current page URL, title, ready state + screenshot [tab_id] Capture page screenshot + text [tab_id] Extract page text content + markdown [tab_id] Extract page as markdown + links [tab_id] Extract all links + snap [tab_id] Get accessibility snapshot (interactive refs) + +Advanced (see references/ADVANCED.md for full docs): + action click [tab_id] + action type [tab_id] + action fill [tab_id] + action select [tab_id] + action scroll [direction] [amount] [tab_id] + action evaluate [tab_id] + action cookies [tab_id] + action audit [tab_id] + action har|har-start|har-stop [tab_id] + action back|forward|reload [tab_id] + action close-tab + +Utilities: + advanced Print path to full API reference +`); + return; + } + + try { + switch (cmd) { + case "health": + await cmdHealth(); + break; + case "tabs": + await cmdTabs(); + break; + case "tab-new": + await cmdTabNew(args[0]); + break; + case "navigate": + await cmdNavigate(args[0], args[1]); + break; + case "page-info": + await cmdPageInfo(args[0]); + break; + case "screenshot": + await cmdScreenshot(args[0], process.env.KURI_OUTPUT); + break; + case "text": + await cmdText(args[0]); + break; + case "markdown": + await cmdMarkdown(args[0]); + break; + case "links": + await cmdLinks(args[0]); + break; + case "snap": + await cmdSnap(args[0]); + break; + case "action": + // Parse remaining args: action [key=value...] + const sub = args[0]; + const actionArgs = {}; + const actionSchemas = { + // Schema: [paramName, ...] for positional filling + // null means all remaining become expression + click: ["ref"], + type: ["ref", "value"], + fill: ["ref", "value"], + select: ["ref", "value"], + scroll: ["direction", "amount"], + evaluate: null, + "close-tab": ["tab_id"], + }; + for (let i = 1; i < args.length; i++) { + if (args[i].includes("=")) { + const [k, ...v] = args[i].split("="); + actionArgs[k] = v.join("="); + } else if (actionSchemas[sub]) { + // Positional: fill schema slots in order + const slot = actionSchemas[sub][Object.keys(actionArgs).length]; + if (slot) actionArgs[slot] = args[i]; + } else if (actionSchemas[sub] === null) { + // evaluate: join all remaining as expression + actionArgs.expression = args.slice(1).join(" "); + break; + } + } + await cmdAction(sub, actionArgs); + break; + case "advanced": + cmdAdvanced(); + break; + default: + console.error(`Unknown command: ${cmd}`); + console.error("Run 'node scripts/kuri.js --help' for usage."); + process.exit(1); + } + } catch (err) { + if (err?.code === "KURI_NOT_RUNNING") { + printKuriNotRunning(err.originalMessage || err.message); + } else { + console.error(`Error: ${err.message}`); + } + process.exit(1); + } +} + +main(); diff --git a/skills/pi-kuri-skills.md b/skills/pi-kuri-skills.md new file mode 100644 index 0000000..948e5b9 --- /dev/null +++ b/skills/pi-kuri-skills.md @@ -0,0 +1,161 @@ +--- +name: pi-kuri-skills +description: Kuri browser automation for pi.dev agents โ€” navigate, screenshot, extract content, and interact with web pages via the Kuri HTTP server. Use when an agent needs to browse the web, read pages, take screenshots, or perform browser interactions. Commands use the `kuri-skill` CLI from `skills/pi-kuri-plugin/`. For advanced operations (click, type, scroll, etc.) read `skills/pi-kuri-advanced.md`. +--- + +# Pi Kuri Skills + +Kuri is a Zig-native browser automation server. This skill uses the +`kuri-skill` CLI wrapper from `skills/pi-kuri-plugin/` to drive Kuri's +HTTP API โ€” no curl, no manual HTTP calls. + +If `kuri-skill` is not available globally, install it: + +```bash +cd skills/pi-kuri-plugin +npm install +npm link # makes `kuri-skill` available on PATH +# or +node scripts/kuri.js +``` + +This skill has two tiers: + +| Tier | Covered here | When to use | +|------|-------------|-------------| +| **Core** ๐ŸŸข | Server health check, start, navigate, screenshot, page info, text & markdown extraction, links, accessibility snapshots | Most tasks | +| **Advanced** ๐Ÿ”ต | Click, type, fill, select, scroll, JS eval, cookies, audit, HAR, session mgmt | Read `skills/pi-kuri-advanced.md` on demand | + +--- + +## 1. Setup โ€” Validate & Start + +### Check if Kuri is running + +```bash +kuri-skill health +``` + +A healthy server returns `{"ok":true}` with the server version. + +If the health check fails โ€” connection refused or timeout โ€” the server is not running. + +### Start Kuri + +If you built from source: + +```bash +# From the kuri repo root +zig build +./zig-out/bin/kuri +``` + +If you installed via the install script: + +```bash +kuri +``` + +Kuri starts a Chrome instance (managing it automatically) and listens on +`http://127.0.0.1:8080` by default. Customize with `HOST`, `PORT`, `CDP_URL` +(see the configuration table in the README). + +### List available tabs + +```bash +kuri-skill tabs +``` + +Returns the active session's tabs with their IDs, URLs, and titles. + +--- + +## 2. Core Operations + +### Create a tab and navigate + +```bash +kuri-skill tab-new https://example.com +kuri-skill navigate https://example.com +``` + +### Get page info + +```bash +kuri-skill page-info +``` + +Returns `url`, `title`, and page `readyState`. Always call this after navigation +to confirm the page loaded before taking further actions. + +### Screenshot + +```bash +kuri-skill screenshot +``` + +Saves a PNG. Customize the output path with `KURI_OUTPUT`. + +### Content extraction + +```bash +kuri-skill text +kuri-skill markdown +kuri-skill links +``` + +### Accessibility snapshot (interactive refs) + +```bash +kuri-skill snap +``` + +Returns a compact text-tree of interactive elements with refs like `e0`, `e1`. +These refs are used as targets for advanced actions (click, type, fill, etc.). + +**Re-snap after any navigation or DOM change** โ€” refs are snapshot-local and +become stale once the page updates. + +--- + +## 3. Advanced Operations + +For click, type, fill, select, scroll, JavaScript evaluation, cookies, security +audits, HAR recording, and session management, read: + +``` +skills/pi-kuri-advanced.md +``` + +**What's available at a glance:** + +| Category | Capabilities | +|----------|-------------| +| ๐Ÿ–ฑ๏ธ Mouse | click, right-click, double-click, hover | +| โŒจ๏ธ Keyboard | type text, fill input, select option | +| ๐Ÿ“œ Scroll | up, down, left, right, pixel-amount | +| ๐Ÿง  JS Eval | arbitrary JavaScript, return values | +| ๐Ÿช Cookies | list, inspect security flags | +| ๐Ÿ”’ Security | audit headers, cookies, HTTPS, mixed content | +| ๐Ÿ“ก HAR | HTTP archive recording (start, stop, export) | +| ๐Ÿ“‘ Tabs & Sessions | multi-tab, multi-session orchestration | +| ๐Ÿงช Experimental | kuri-browser render, bench, parity, serve-cdp | + +Read the advanced file only when you need one of those capabilities. + +--- + +## Tips + +- **Prefer `kuri-skill` with sessions** โ€” set `KURI_SESSION` env var to group + tabs without repeating `tab_id` on every call. +- **Call `kuri-skill page-info` before acting** โ€” confirm the right page is loaded. +- **Treat refs as snapshot-local** โ€” refresh after navigation or DOM updates. +- **`KURI_ALLOW_PRIVATE=1`** โ€” if you need to reach localhost or private IPs + during development, set this before starting the Kuri server. + +## Plugin reference + +- `skills/pi-kuri-plugin/` โ€” the npm-installable plugin with `kuri-skill` CLI +- `skills/pi-kuri-plugin/package.json` โ€” `pi.skills` field for pi auto-discovery +- `skills/pi-kuri-plugin/SKILL.md` โ€” pi agent skill (synchronized with this file)