From 8f567fd121343199b51a7677598c556a7544b392 Mon Sep 17 00:00:00 2001 From: Robert Maefs Date: Mon, 1 Jun 2026 13:37:11 -0400 Subject: [PATCH 1/8] =?UTF-8?q?feat(skills):=20add=20pi.dev=20two-tier=20s?= =?UTF-8?q?kill=20=E2=80=94=20pi-kuri-skills=20+=20pi-kuri-advanced?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add a pair of pi.dev-format skill files alongside the existing upstream kuri-skill.md. The new files follow a two-tier design that keeps agent context lean while making all capabilities discoverable. pi-kuri-skills.md (core): - Section 1: Setup โ€” health check, server start instructions, tab listing - Section 2: Core operations โ€” navigate, page-info, screenshot, text, markdown, links, accessibility snapshot with interactive refs - Section 3: Compact capability table hinting at advanced companion - Tips, browser path reference, custom project skills guidance pi-kuri-advanced.md (companion reference): - Mouse: click, right-click, double-click, hover - Keyboard: type, fill, select - Scrolling with direction and amount control - JavaScript evaluation with common expression examples - Cookies listing with security flags - Security audit (headers, cookies, HTTPS, mixed content) - HAR recording for API discovery - Tab and session management, multi-session orchestration - Experimental kuri-browser CLI documentation - Four common workflow patterns with full curl examples - Quick parameter reference table and full endpoint map (~80 endpoints) README updated to reference both new files. --- readme.md | 2 + skills/pi-kuri-advanced.md | 511 +++++++++++++++++++++++++++++++++++++ skills/pi-kuri-skills.md | 182 +++++++++++++ 3 files changed, 695 insertions(+) create mode 100644 skills/pi-kuri-advanced.md create mode 100644 skills/pi-kuri-skills.md diff --git a/readme.md b/readme.md index 42c043c..e6dc65d 100644 --- a/readme.md +++ b/readme.md @@ -502,6 +502,8 @@ The lowest-friction server loop is: The repo includes a user-extensible skill area: - `skills/kuri-skill.md` is the base Kuri HTTP-agent skill +- `skills/pi-kuri-skills.md` is a two-tier pi.dev skill (core operations + compact advanced hint) +- `skills/pi-kuri-advanced.md` is the companion advanced reference for pi.dev (click, type, scroll, JS eval, cookies, audit, HAR, etc.) - `skills/custom/` is reserved for your own project-specific skills - `skills/custom/hackernews-page-2.md` is a concrete example custom skill - `.claude/skills/kuri-server/SKILL.md` stays in sync for Claude-style repo skills diff --git a/skills/pi-kuri-advanced.md b/skills/pi-kuri-advanced.md new file mode 100644 index 0000000..a9a5906 --- /dev/null +++ b/skills/pi-kuri-advanced.md @@ -0,0 +1,511 @@ +--- +name: pi-kuri-advanced +description: Advanced Kuri browser automation for pi.dev agents โ€” click, type, fill, select, scroll, JavaScript evaluation, cookies, security audits, session management, HAR recording, and the experimental kuri-browser CLI. Load this on demand when the core skill (`skills/pi-kuri-skills.md`) is not enough. +--- + +# Pi Kuri Advanced Reference + +This file covers operations beyond the core skill (`skills/pi-kuri-skills.md`). +Load it when you need interactive actions, JavaScript evaluation, cookies, +security audits, session management, or experimental features. + +--- + +## Using `X-Kuri-Session` + +Most examples below use the `X-Kuri-Session` header, which groups tabs and tracks +a current tab per session so you don't need to repeat `tab_id` on every call. + +```bash +BASE=http://127.0.0.1:8080 +SESSION=my-session + +# All subsequent calls use this session +curl -s -H "X-Kuri-Session: $SESSION" "$BASE/action?action=click&ref=e1" +``` + +If you need to target a specific tab, add `&tab_id=TABID123` to any endpoint. +Use `/tabs` to discover tab IDs. + +--- + +## ๐Ÿ–ฑ๏ธ Mouse Actions + +### Click an element + +```bash +# Click by snapshot ref +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/action?action=click&ref=e1" + +# Click with a specific tab +curl -s "http://127.0.0.1:8080/action?action=click&ref=e1&tab_id=TABID123" + +# Click with modifiers (shift, ctrl, meta) +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/action?action=click&ref=e1&modifiers=shift" +``` + +### Right-click + +```bash +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/action?action=click&ref=e1&button=right" +``` + +### Double-click + +```bash +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/action?action=click&ref=e1&clickCount=2" +``` + +### Hover + +Use the snapshot to detect hover-triggered UI, then evaluate JS for precise +hover scenarios: + +```bash +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/evaluate?expression=document.querySelector('button').dispatchEvent(new MouseEvent('mouseover',{bubbles:true}))" +``` + +--- + +## โŒจ๏ธ Keyboard Actions + +### Type text into an input + +Replaces existing content with the provided text: + +```bash +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/action?action=type&ref=e2&value=hello%20world" +``` + +### Fill input value + +Sets the input's value directly (similar to type, may clear first): + +```bash +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/action?action=fill&ref=e2&value=user@example.com" +``` + +### Select a dropdown option + +```bash +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/action?action=select&ref=e3&value=option-value" +``` + +--- + +## ๐Ÿ“œ Scrolling + +```bash +# Scroll down (default) +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/action?action=scroll&direction=down" + +# Scroll up with specified amount +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/action?action=scroll&direction=up&amount=10" + +# Scroll right (for horizontally scrollable content) +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/action?action=scroll&direction=right&amount=3" +``` + +The `amount` parameter controls distance (lines or approximate pixels). + +--- + +## ๐Ÿง  JavaScript Evaluation + +### Execute JavaScript in the page context + +```bash +# Get page title +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/evaluate?expression=document.title" + +# Read heading text +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/evaluate?expression=document.querySelector('h1').textContent" + +# Extract dynamic state +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/evaluate?expression=JSON.stringify(window.__INITIAL_STATE__)" + +# Get page metrics +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/evaluate?expression=document.body.scrollHeight" + +# Check element visibility +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/evaluate?expression=document.querySelector('.spinner').offsetParent!==null" +``` + +Returns the evaluated result as a string, number, boolean, or JSON value. + +--- + +## ๐Ÿช Cookies + +List all cookies for the current page with security flags: + +```bash +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/cookies" +``` + +Returns an array of cookies with: `name`, `value`, `domain`, `path`, `secure`, +`httpOnly`, `sameSite`, `expires`. + +--- + +## ๐Ÿ”’ Security Audit + +Run a full security audit on the current page: + +```bash +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/audit" +``` + +Checks performed: +- Security response headers: HSTS, CSP, X-Frame-Options, X-Content-Type-Options +- Cookie security: Secure flag, HttpOnly flag, SameSite attribute +- HTTPS enforcement +- Mixed content warnings + +--- + +## ๐Ÿ“ก HAR Recording (HTTP Archive) + +Record network activity for API discovery or debugging: + +```bash +# Start recording +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/har/start" + +# Perform actions that make network requests... + +# Stop and retrieve the HAR log +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/har/stop" + +# Get current HAR log without stopping +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/har" +``` + +Useful for reverse-engineering API calls, finding XHR endpoints, or debugging +network issues. + +--- + +## ๐Ÿ“‘ Tab & Session Management + +### List all tabs + +```bash +curl -s "http://127.0.0.1:8080/tabs" +``` + +### Create a new tab + +```bash +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/tab/new?url=https%3A%2F%2Fexample.com" +``` + +### Close a tab + +```bash +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/tab/close?tab_id=TABID123" +``` + +### Set current tab + +```bash +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/tab/current?tab_id=TABID123" +``` + +### Browser navigation + +```bash +# Back +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/back" + +# Forward +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/forward" + +# Reload +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/reload" +``` + +### Multiple sessions + +Run independent browser sessions in parallel by using different session IDs: + +```bash +curl -s -H "X-Kuri-Session: session-A" "http://127.0.0.1:8080/tab/new?url=https%3A%2F%2Fexample.com" +curl -s -H "X-Kuri-Session: session-B" "http://127.0.0.1:8080/tab/new?url=https%3A%2F%2Fother.com" +``` + +Each session tracks its own tabs and current tab independently. + +--- + +## ๐Ÿงช Experimental โ€” kuri-browser + +The `kuri-browser/` subdirectory is an **experimental** Zig-native browser +runtime with its own rendering engine. It is a separate build and is not wired +into Kuri's root build. Use it for evaluation and development only. + +### Build and run + +```bash +cd kuri-browser +zig build run -- render https://news.ycombinator.com --selector ".titleline a" --dump text +zig build run -- render https://todomvc.com/examples/react/dist/ --js --wait-eval "document.querySelectorAll('.todo-list li').length >= 1" +zig build run -- bench --offline +zig build run -- parity --offline +zig build run -- serve-cdp --port 9333 +``` + +`serve-cdp` exposes a minimal CDP-compatible WebSocket server useful for +protocol smoke tests, but it is not Playwright/Puppeteer-compatible enough +to replace Chrome. + +### Screenshot via kuri-browser + +Screenshots currently fall back to the main Kuri/CDP renderer, so start the +normal Kuri server first: + +```bash +# terminal 1 โ€” start main Kuri +zig build +./zig-out/bin/kuri + +# terminal 2 โ€” take screenshot via kuri-browser +cd kuri-browser +zig build run -- screenshot https://example.com --out example.jpg \ + --compress --kuri-base http://127.0.0.1:8080 +``` + +`--compress` captures PNG and JPEG, writes the smaller file, and reports +size savings. + +--- + +## Common Workflow Patterns + +### Pattern 1: Quick screenshot verification + +```bash +SESSION=verify + +# Open page +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/tab/new?url=https%3A%2F%2Fexample.com" + +# Confirm loaded +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/page/info" + +# Capture visual evidence +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/screenshot" -o evidence.png +``` + +### Pattern 2: Fill form and submit + +```bash +SESSION=login + +# Open login page +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/tab/new?url=https%3A%2F%2Fexample.com%2Flogin" + +# Get snapshot for refs +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/snapshot?filter=interactive&format=compact" + +# Fill credentials (use refs from snapshot) +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/action?action=type&ref=e0&value=user@example.com" +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/action?action=type&ref=e1&value=password123" + +# Click submit button +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/action?action=click&ref=e2" + +# Verify result +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/page/info" +``` + +### Pattern 3: Extract dynamic content + +```bash +SESSION=extract + +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/tab/new?url=https%3A%2F%2Fexample.com%2Fapp" + +# Extract JS state +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/evaluate?expression=JSON.stringify(window.__DATA__)" + +# Also get rendered text +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/text" +``` + +### Pattern 4: Multi-step interaction loop + +```bash +SESSION=search + +# 1. Open page +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/tab/new?url=https%3A%2F%2Fexample.com%2Fsearch" + +# 2. Get snapshot for refs +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/snapshot?filter=interactive&format=compact" + +# 3. Type search query +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/action?action=type&ref=e0&value=query+text" + +# 4. Wait and check for results via JS +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/evaluate?expression=document.querySelectorAll('.result').length" + +# 5. Re-snap after DOM change +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/snapshot?filter=interactive&format=compact" + +# 6. Click a result +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/action?action=click&ref=e3" + +# 7. Verify new page +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/page/info" + +# 8. Screenshot for evidence +curl -s -H "X-Kuri-Session: $SESSION" \ + "http://127.0.0.1:8080/screenshot" -o result.png +``` + +--- + +## Quick Parameter Reference + +### Common query parameters + +| Parameter | Type | Applies to | Description | +|-----------|------|-----------|-------------| +| `tab_id` | string | Most endpoints | Target a specific tab | +| `ref` | string | `/action` | Element reference from snapshot | +| `action` | string | `/action` | `click`, `type`, `fill`, `select`, `scroll` | +| `value` | string | type / fill / select | Text or option value | +| `direction` | string | scroll | `up`, `down`, `left`, `right` | +| `amount` | number | scroll | Scroll distance (lines or pixels) | +| `button` | string | click | `left`, `right`, `middle` | +| `clickCount` | number | click | `1` = single, `2` = double | +| `modifiers` | string | click | `shift`, `ctrl`, `meta` (comma-separated) | +| `expression` | string | `/evaluate` | JavaScript to execute | +| `filter` | string | `/snapshot` | `interactive`, `all` | +| `format` | string | `/snapshot` | `compact`, `json`, `verbose` | +| `session` | string | Most endpoints | `X-Kuri-Session` header override | +| `wait` | number | Most endpoints | Milliseconds to wait before response | +| `timeout` | number | Most endpoints | Operation timeout in milliseconds | +| `url` | string | `/navigate`, `/tab/new` | Target URL (URL-encoded) | + +### Full endpoint map + +| Method | Endpoint | Purpose | +|--------|----------|---------| +| GET | `/health` | Server health check | +| GET | `/tabs` | List all tabs | +| GET/POST | `/tab/new` | Create a new tab | +| GET | `/tab/close` | Close a tab | +| GET | `/tab/current` | Set or get current tab | +| GET | `/navigate` | Navigate a tab to a URL | +| GET | `/page/info` | Current page metadata (URL, title, state) | +| GET | `/page/state` | Lightweight page observation (48 tokens) | +| GET | `/screenshot` | Capture viewport screenshot | +| GET | `/snapshot` | Accessibility tree snapshot | +| GET | `/text` | Page plain text | +| GET | `/markdown` | Page as markdown | +| GET | `/links` | Extract all links | +| GET | `/action` | Perform action (click, type, fill, select, scroll) | +| GET | `/evaluate` | Execute JavaScript | +| GET | `/cookies` | List cookies with security flags | +| GET | `/audit` | Full security audit | +| GET | `/back` | Browser back | +| GET | `/forward` | Browser forward | +| GET | `/reload` | Reload page | +| GET | `/har` | Current HAR log | +| GET | `/har/start` | Start HAR recording | +| GET | `/har/stop` | Stop and return HAR | +| GET | `/headers` | Response security headers | +| GET | `/token` | Print or rotate API token | +| GET | `/discover` | Discover CDP tabs | +| POST | `/batch` | Execute multiple commands in one call | +| GET | `/element/state` | Quick element state check | +| GET | `/find-element` | Semantic locator search | +| GET | `/clipboard/read` | Read clipboard content | +| GET | `/clipboard/write` | Write to clipboard | +| GET | `/dialog/auto` | Auto-accept dialogs | +| GET | `/dialog/accept` | Accept current dialog | +| GET | `/dialog/dismiss` | Dismiss current dialog | +| GET | `/mouse/move` | Move mouse to coordinates | +| GET | `/mouse/down` | Mouse button down | +| GET | `/mouse/up` | Mouse button up | +| GET | `/mouse/wheel` | Mouse wheel scroll | +| GET | `/timezone` | Set emulated timezone | +| GET | `/locale` | Set emulated locale | +| GET | `/permissions` | Set browser permissions | +| GET | `/clear` | Clear input content | +| GET | `/selectall` | Select all in input | +| GET | `/boundingbox` | Element bounding box | +| GET | `/getattribute` | Get element attribute | +| GET | `/inputvalue` | Get input value | +| GET | `/evalhandle` | Evaluate and return JS handle | +| GET | `/recording/start` | Start action recording | +| GET | `/recording/stop` | Stop and export recording | +| GET | `/recording/export` | Export recording as batch JSON | +| GET | `/vitals` | Core Web Vitals (LCP, CLS, FID, TTFB, FCP) | +| GET | `/pushstate` | Simulate pushState navigation | +| GET | `/bringtofront` | Bring tab to front | +| GET | `/frame` | Switch iframe context | +| GET | `/mainframe` | Switch to main frame | +| GET | `/addstyle` | Inject CSS into page | +| GET | `/expose` | Expose Node.js function to page | +| GET | `/setcontent` | Set page HTML content | +| GET | `/request/detail` | Request detail by ID | +| GET | `/response/body` | Response body by request ID | +| GET | `/download` | Download a file | +| GET | `/diff/url` | Compare two URLs side by side | +| GET | `/cache/set` | Cache a value | +| GET | `/cache/get` | Get cached value | +| GET | `/cache/clear` | Clear cache | +| GET | `/cache/list` | List cache keys | +| GET | `/screenshot/som` | Set-of-Marks screenshot with numbered overlays | +| GET | `/snapshot/changes` | Smart diff โ€” only changed lines since last snapshot | + +See the main README for a complete, always-up-to-date endpoint list. diff --git a/skills/pi-kuri-skills.md b/skills/pi-kuri-skills.md new file mode 100644 index 0000000..e58516b --- /dev/null +++ b/skills/pi-kuri-skills.md @@ -0,0 +1,182 @@ +--- +name: pi-kuri-skills +description: Kuri browser automation for pi.dev agents โ€” navigate, screenshot, extract content, and interact with web pages via the Kuri HTTP server. Use when an agent needs to browse the web, read pages, take screenshots, or perform browser interactions. This is the pi.dev core skill โ€” read skills/pi-kuri-advanced.md for click, type, fill, scroll, JavaScript evaluation, cookies, security audits, and more. +--- + +# Pi Kuri Skills + +Kuri is a Zig-native browser automation server. Use its HTTP API for agent-driven +browser automation โ€” no Node.js, no Playwright, no Python dependencies. + +This skill has two tiers: + +| Tier | Covered here | When to use | +|------|-------------|-------------| +| **Core** ๐ŸŸข | Server health check, start/stop, navigate, screenshot, page info, text & markdown extraction, links, accessibility snapshots | Most tasks | +| **Advanced** ๐Ÿ”ต | Click, type, fill, select, scroll, JS eval, cookies, audit, HAR recording, tab management, session control, experimental kuri-browser | Read `skills/pi-kuri-advanced.md` on demand when you need them | + +--- + +## 1. Setup โ€” Validate & Start + +### Check if Kuri is running + +```bash +curl -s http://127.0.0.1:8080/health +``` + +A healthy server returns `{"ok":true}` with the server version. + +If the health check fails โ€” connection refused or timeout โ€” the server is not running. + +### Start Kuri + +If you built from source: + +```bash +# From the repo root +zig build +./zig-out/bin/kuri +``` + +If you installed via the script: + +```bash +kuri +``` + +Kuri starts a Chrome instance (managing it automatically) and listens on +`http://127.0.0.1:8080` by default. Customize with `HOST`, `PORT`, `CDP_URL` +(see the configuration table in the README). + +### List available tabs + +```bash +curl -s http://127.0.0.1:8080/tabs +``` + +Returns the active session's tabs with their IDs, URLs, and titles. + +--- + +## 2. Core Operations + +All examples use `curl` against the default server at `http://127.0.0.1:8080`. +Set `BASE=http://127.0.0.1:8080` and `SESSION=my-session` for reuse. + +### Create a tab and navigate + +```bash +# Open a new tab (session-scoped with X-Kuri-Session header) +curl -s -H "X-Kuri-Session: my-session" \ + "http://127.0.0.1:8080/tab/new?url=https%3A%2F%2Fexample.com" +``` + +### Get page info + +```bash +curl -s -H "X-Kuri-Session: my-session" \ + "http://127.0.0.1:8080/page/info" +``` + +Returns `url`, `title`, and page `readyState`. Always call this after navigation +to confirm the page loaded before taking further actions. + +### Screenshot + +```bash +curl -s -H "X-Kuri-Session: my-session" \ + "http://127.0.0.1:8080/screenshot" -o screenshot.png +``` + +Returns a PNG. Add `?format=jpeg&quality=80` for smaller files. + +### Content extraction + +```bash +# Plain text +curl -s -H "X-Kuri-Session: my-session" \ + "http://127.0.0.1:8080/text" + +# Markdown +curl -s -H "X-Kuri-Session: my-session" \ + "http://127.0.0.1:8080/markdown" + +# All links +curl -s -H "X-Kuri-Session: my-session" \ + "http://127.0.0.1:8080/links" +``` + +### Accessibility snapshot (interactive refs) + +```bash +curl -s -H "X-Kuri-Session: my-session" \ + "http://127.0.0.1:8080/snapshot?filter=interactive&format=compact" +``` + +Returns a compact text-tree of interactive elements with refs like `@e0`, `@e1`. +These refs are used as targets for advanced actions (click, type, fill, etc.). + +**Re-snap after any navigation or DOM change** โ€” refs are snapshot-local and +become stale once the page updates. + +--- + +## 3. Advanced Operations + +Kuri supports many more operations: click, right-click, double-click, hover, +type, fill, select dropdowns, scroll, JavaScript evaluation, cookies, +security audits, HAR recording, tab and session management, and the +experimental `kuri-browser` CLI. + +For full details, read: + +``` +skills/pi-kuri-advanced.md +``` + +**What's in the advanced file at a glance:** + +| Category | Capabilities | +|----------|-------------| +| ๐Ÿ–ฑ๏ธ Mouse | click, right-click, double-click, hover | +| โŒจ๏ธ Keyboard | type text, fill input, select option, press keys | +| ๐Ÿ“œ Scroll | up, down, left, right, pixel-amount, element-scroll | +| ๐Ÿง  JS Eval | arbitrary JavaScript, return values | +| ๐Ÿช Cookies | list, inspect security flags | +| ๐Ÿ”’ Security | audit headers, cookies, HTTPS, mixed content | +| ๐Ÿ“ก HAR | HTTP archive recording (start, stop, export) | +| ๐Ÿ“‘ Tabs | create, close, list, switch, multi-tab workflows | +| ๐Ÿ”„ Sessions | X-Kuri-Session grouping, multi-session orchestration | +| ๐Ÿงช Experimental | kuri-browser render, bench, parity, serve-cdp | + +Read the advanced file only when you need one of those capabilities. The +detailed instructions, endpoint signatures, and workflow examples live there. + +--- + +## Tips + +- **Prefer `X-Kuri-Session`** over repeating `tab_id` โ€” sessions group tabs + and Kuri tracks the current tab for you. +- **Call `/page/info` before acting** โ€” confirm the right page is loaded. +- **Use `filter=interactive&format=compact`** for snapshot in agent loops. + Reserve `format=json` for programmatic parsing. +- **Read snapshot `state`** before acting on controls โ€” check for + `checked`, `disabled`, `readonly`, `expanded`, `selected` properties. +- **Treat refs as snapshot-local** โ€” refresh after navigation or DOM updates. +- **`KURI_ALLOW_PRIVATE=1`** โ€” if you need to navigate to localhost or + private IPs during development, set this env var before starting the server. + +## Browser path reference + +- **`kuri` (HTTP API)** โ€” production path for agent loops (this skill). +- **`kuri-fetch`** โ€” standalone no-Chrome text extraction from URLs. +- **`kuri-browse`** โ€” interactive terminal browser. +- **`kuri-agent`** โ€” scriptable CLI automation against the Kuri server. +- **`kuri-browser/`** โ€” experimental Zig-native browser runtime (separate build). + +## Custom project skills + +Put site-specific workflows (login steps, selectors, data extraction patterns) +in `skills/custom/`. See `skills/custom/hackernews-page-2.md` for an example. From 0ec67c1d9c965889fadfd948c35be81a242512f8 Mon Sep 17 00:00:00 2001 From: Robert Maefs Date: Mon, 1 Jun 2026 13:44:03 -0400 Subject: [PATCH 2/8] =?UTF-8?q?feat(skills):=20add=20pi-kuri-plugin=20?= =?UTF-8?q?=E2=80=94=20npm-installable=20pi.dev=20skill=20package?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add a standalone pi.dev plugin directory with everything needed for npm distribution and pi agent auto-discovery: pi-kuri-plugin/ โ”œโ”€โ”€ SKILL.md # Pi agent skill (two-tier design) โ”œโ”€โ”€ package.json # npm metadata + pi.skills + bin entry โ”œโ”€โ”€ README.md # Install/use docs โ”œโ”€โ”€ scripts/ โ”‚ โ””โ”€โ”€ kuri.js # CLI wrapper (bin: kuri-skill) โ””โ”€โ”€ references/ โ””โ”€โ”€ ADVANCED.md # On-demand advanced reference Key features: - bin entry: 'kuri-skill' command available after npm install -g - pi.skills field for auto-discovery by pi agent - No external dependencies (uses Node 18+ built-in fetch) - Disentangled from mh/milhouse โ€” uses kuri binary directly - SKILL.md content synchronized with pi-kuri-skills.md - ADVANCED.md content synchronized with pi-kuri-advanced.md - README documents install, config, and quick start --- readme.md | 1 + skills/pi-kuri-advanced.md | 441 +++++-------- skills/pi-kuri-plugin/.npmignore | 2 + skills/pi-kuri-plugin/README.md | 144 +++++ skills/pi-kuri-plugin/SKILL.md | 176 ++++++ skills/pi-kuri-plugin/package.json | 22 + skills/pi-kuri-plugin/references/ADVANCED.md | 354 +++++++++++ skills/pi-kuri-plugin/scripts/kuri.js | 622 +++++++++++++++++++ skills/pi-kuri-skills.md | 113 ++-- 9 files changed, 1509 insertions(+), 366 deletions(-) create mode 100644 skills/pi-kuri-plugin/.npmignore create mode 100644 skills/pi-kuri-plugin/README.md create mode 100644 skills/pi-kuri-plugin/SKILL.md create mode 100644 skills/pi-kuri-plugin/package.json create mode 100644 skills/pi-kuri-plugin/references/ADVANCED.md create mode 100755 skills/pi-kuri-plugin/scripts/kuri.js diff --git a/readme.md b/readme.md index e6dc65d..ddde92d 100644 --- a/readme.md +++ b/readme.md @@ -504,6 +504,7 @@ The repo includes a user-extensible skill area: - `skills/kuri-skill.md` is the base Kuri HTTP-agent skill - `skills/pi-kuri-skills.md` is a two-tier pi.dev skill (core operations + compact advanced hint) - `skills/pi-kuri-advanced.md` is the companion advanced reference for pi.dev (click, type, scroll, JS eval, cookies, audit, HAR, etc.) +- `skills/pi-kuri-plugin/` is a standalone npm-installable pi.dev plugin with CLI wrapper (`kuri-skill`), package.json, and skill files - `skills/custom/` is reserved for your own project-specific skills - `skills/custom/hackernews-page-2.md` is a concrete example custom skill - `.claude/skills/kuri-server/SKILL.md` stays in sync for Claude-style repo skills diff --git a/skills/pi-kuri-advanced.md b/skills/pi-kuri-advanced.md index a9a5906..8d9ff81 100644 --- a/skills/pi-kuri-advanced.md +++ b/skills/pi-kuri-advanced.md @@ -9,24 +9,19 @@ This file covers operations beyond the core skill (`skills/pi-kuri-skills.md`). Load it when you need interactive actions, JavaScript evaluation, cookies, security audits, session management, or experimental features. ---- +All commands use the `kuri-skill` CLI from `skills/pi-kuri-plugin/`. If +`kuri-skill` is not on your PATH, run commands via: -## Using `X-Kuri-Session` +```bash +node skills/pi-kuri-plugin/scripts/kuri.js +``` -Most examples below use the `X-Kuri-Session` header, which groups tabs and tracks -a current tab per session so you don't need to repeat `tab_id` on every call. +Set `KURI_SESSION` to group tabs without repeating `tab_id` on every call: ```bash -BASE=http://127.0.0.1:8080 -SESSION=my-session - -# All subsequent calls use this session -curl -s -H "X-Kuri-Session: $SESSION" "$BASE/action?action=click&ref=e1" +export KURI_SESSION=my-session ``` -If you need to target a specific tab, add `&tab_id=TABID123` to any endpoint. -Use `/tabs` to discover tab IDs. - --- ## ๐Ÿ–ฑ๏ธ Mouse Actions @@ -34,30 +29,14 @@ Use `/tabs` to discover tab IDs. ### Click an element ```bash -# Click by snapshot ref -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/action?action=click&ref=e1" - -# Click with a specific tab -curl -s "http://127.0.0.1:8080/action?action=click&ref=e1&tab_id=TABID123" +kuri-skill action click e1 +kuri-skill action click ref=e1 tab_id=TABID123 -# Click with modifiers (shift, ctrl, meta) -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/action?action=click&ref=e1&modifiers=shift" -``` - -### Right-click +# Right-click +kuri-skill action click ref=e1 button=right -```bash -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/action?action=click&ref=e1&button=right" -``` - -### Double-click - -```bash -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/action?action=click&ref=e1&clickCount=2" +# Double-click +kuri-skill action click ref=e1 clickCount=2 ``` ### Hover @@ -66,8 +45,8 @@ Use the snapshot to detect hover-triggered UI, then evaluate JS for precise hover scenarios: ```bash -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/evaluate?expression=document.querySelector('button').dispatchEvent(new MouseEvent('mouseover',{bubbles:true}))" +kuri-skill action evaluate \ + expression="document.querySelector('button').dispatchEvent(new MouseEvent('mouseover',{bubbles:true}))" ``` --- @@ -76,27 +55,23 @@ curl -s -H "X-Kuri-Session: $SESSION" \ ### Type text into an input -Replaces existing content with the provided text: - ```bash -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/action?action=type&ref=e2&value=hello%20world" +kuri-skill action type e2 "hello world" +kuri-skill action type ref=e2 value="hello world" ``` ### Fill input value -Sets the input's value directly (similar to type, may clear first): - ```bash -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/action?action=fill&ref=e2&value=user@example.com" +kuri-skill action fill e2 "user@example.com" +kuri-skill action fill ref=e2 value="user@example.com" ``` ### Select a dropdown option ```bash -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/action?action=select&ref=e3&value=option-value" +kuri-skill action select e3 "option-value" +kuri-skill action select ref=e3 value="option-value" ``` --- @@ -105,46 +80,41 @@ curl -s -H "X-Kuri-Session: $SESSION" \ ```bash # Scroll down (default) -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/action?action=scroll&direction=down" +kuri-skill action scroll -# Scroll up with specified amount -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/action?action=scroll&direction=up&amount=10" +# Scroll up +kuri-skill action scroll direction=up amount=10 -# Scroll right (for horizontally scrollable content) -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/action?action=scroll&direction=right&amount=3" -``` +# Scroll right +kuri-skill action scroll direction=right amount=3 -The `amount` parameter controls distance (lines or approximate pixels). +# Scroll by pixel amount +kuri-skill action scroll direction=down amount=500 +``` --- ## ๐Ÿง  JavaScript Evaluation -### Execute JavaScript in the page context - ```bash # Get page title -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/evaluate?expression=document.title" +kuri-skill action evaluate expression="document.title" # Read heading text -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/evaluate?expression=document.querySelector('h1').textContent" +kuri-skill action evaluate \ + expression="document.querySelector('h1').textContent" # Extract dynamic state -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/evaluate?expression=JSON.stringify(window.__INITIAL_STATE__)" +kuri-skill action evaluate \ + expression="JSON.stringify(window.__INITIAL_STATE__)" # Get page metrics -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/evaluate?expression=document.body.scrollHeight" +kuri-skill action evaluate \ + expression="document.body.scrollHeight" # Check element visibility -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/evaluate?expression=document.querySelector('.spinner').offsetParent!==null" +kuri-skill action evaluate \ + expression="document.querySelector('.spinner').offsetParent!==null" ``` Returns the evaluated result as a string, number, boolean, or JSON value. @@ -153,11 +123,8 @@ Returns the evaluated result as a string, number, boolean, or JSON value. ## ๐Ÿช Cookies -List all cookies for the current page with security flags: - ```bash -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/cookies" +kuri-skill action cookies ``` Returns an array of cookies with: `name`, `value`, `domain`, `path`, `secure`, @@ -167,11 +134,8 @@ Returns an array of cookies with: `name`, `value`, `domain`, `path`, `secure`, ## ๐Ÿ”’ Security Audit -Run a full security audit on the current page: - ```bash -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/audit" +kuri-skill action audit ``` Checks performed: @@ -188,77 +152,50 @@ Record network activity for API discovery or debugging: ```bash # Start recording -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/har/start" +kuri-skill action har-start # Perform actions that make network requests... # Stop and retrieve the HAR log -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/har/stop" +kuri-skill action har-stop # Get current HAR log without stopping -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/har" +kuri-skill action har ``` -Useful for reverse-engineering API calls, finding XHR endpoints, or debugging -network issues. +Useful for reverse-engineering API calls or debugging network issues. --- ## ๐Ÿ“‘ Tab & Session Management -### List all tabs - ```bash -curl -s "http://127.0.0.1:8080/tabs" -``` +# List all tabs +kuri-skill tabs -### Create a new tab +# Create a new tab +kuri-skill tab-new https://example.com -```bash -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/tab/new?url=https%3A%2F%2Fexample.com" -``` - -### Close a tab - -```bash -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/tab/close?tab_id=TABID123" -``` - -### Set current tab +# Close a tab +kuri-skill action close-tab TABID123 -```bash -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/tab/current?tab_id=TABID123" -``` - -### Browser navigation - -```bash -# Back -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/back" +# Browser back +kuri-skill action back -# Forward -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/forward" +# Browser forward +kuri-skill action forward # Reload -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/reload" +kuri-skill action reload ``` ### Multiple sessions -Run independent browser sessions in parallel by using different session IDs: +Run independent browser sessions by setting `KURI_SESSION` to different values: ```bash -curl -s -H "X-Kuri-Session: session-A" "http://127.0.0.1:8080/tab/new?url=https%3A%2F%2Fexample.com" -curl -s -H "X-Kuri-Session: session-B" "http://127.0.0.1:8080/tab/new?url=https%3A%2F%2Fother.com" +KURI_SESSION=session-a kuri-skill tab-new https://example.com +KURI_SESSION=session-b kuri-skill tab-new https://other.com ``` Each session tracks its own tabs and current tab independently. @@ -268,43 +205,26 @@ Each session tracks its own tabs and current tab independently. ## ๐Ÿงช Experimental โ€” kuri-browser The `kuri-browser/` subdirectory is an **experimental** Zig-native browser -runtime with its own rendering engine. It is a separate build and is not wired -into Kuri's root build. Use it for evaluation and development only. - -### Build and run +runtime with its own rendering engine (separate build, not wired into Kuri's +root build). Use for evaluation and development only. ```bash cd kuri-browser zig build run -- render https://news.ycombinator.com --selector ".titleline a" --dump text -zig build run -- render https://todomvc.com/examples/react/dist/ --js --wait-eval "document.querySelectorAll('.todo-list li').length >= 1" +zig build run -- render https://todomvc.com/examples/react/dist/ --js --wait-eval "..." zig build run -- bench --offline -zig build run -- parity --offline zig build run -- serve-cdp --port 9333 ``` -`serve-cdp` exposes a minimal CDP-compatible WebSocket server useful for -protocol smoke tests, but it is not Playwright/Puppeteer-compatible enough -to replace Chrome. - -### Screenshot via kuri-browser - -Screenshots currently fall back to the main Kuri/CDP renderer, so start the -normal Kuri server first: +Screenshots fall back to the main Kuri/CDP renderer โ€” start the normal Kuri +server first, then: ```bash -# terminal 1 โ€” start main Kuri -zig build -./zig-out/bin/kuri - -# terminal 2 โ€” take screenshot via kuri-browser cd kuri-browser zig build run -- screenshot https://example.com --out example.jpg \ --compress --kuri-base http://127.0.0.1:8080 ``` -`--compress` captures PNG and JPEG, writes the smaller file, and reports -size savings. - --- ## Common Workflow Patterns @@ -312,200 +232,123 @@ size savings. ### Pattern 1: Quick screenshot verification ```bash -SESSION=verify - -# Open page -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/tab/new?url=https%3A%2F%2Fexample.com" +export KURI_SESSION=verify -# Confirm loaded -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/page/info" - -# Capture visual evidence -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/screenshot" -o evidence.png +kuri-skill tab-new https://example.com +kuri-skill page-info +kuri-skill screenshot # set KURI_OUTPUT to customize path ``` ### Pattern 2: Fill form and submit ```bash -SESSION=login - -# Open login page -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/tab/new?url=https%3A%2F%2Fexample.com%2Flogin" - -# Get snapshot for refs -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/snapshot?filter=interactive&format=compact" +export KURI_SESSION=login -# Fill credentials (use refs from snapshot) -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/action?action=type&ref=e0&value=user@example.com" -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/action?action=type&ref=e1&value=password123" +kuri-skill tab-new https://example.com/login +kuri-skill snap # Get refs -# Click submit button -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/action?action=click&ref=e2" +kuri-skill action type e0 "user@example.com" +kuri-skill action type e1 "password123" +kuri-skill action click e2 # Submit button -# Verify result -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/page/info" +kuri-skill page-info # Check result ``` ### Pattern 3: Extract dynamic content ```bash -SESSION=extract - -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/tab/new?url=https%3A%2F%2Fexample.com%2Fapp" - -# Extract JS state -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/evaluate?expression=JSON.stringify(window.__DATA__)" +export KURI_SESSION=extract -# Also get rendered text -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/text" +kuri-skill tab-new https://example.com/app +kuri-skill action evaluate expression="JSON.stringify(window.__DATA__)" +kuri-skill text # Also get rendered text ``` ### Pattern 4: Multi-step interaction loop ```bash -SESSION=search +export KURI_SESSION=search # 1. Open page -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/tab/new?url=https%3A%2F%2Fexample.com%2Fsearch" +kuri-skill tab-new https://example.com/search # 2. Get snapshot for refs -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/snapshot?filter=interactive&format=compact" +kuri-skill snap # 3. Type search query -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/action?action=type&ref=e0&value=query+text" +kuri-skill action type e0 "query text" -# 4. Wait and check for results via JS -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/evaluate?expression=document.querySelectorAll('.result').length" +# 4. Check for results via JS +kuri-skill action evaluate \ + expression="document.querySelectorAll('.result').length" # 5. Re-snap after DOM change -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/snapshot?filter=interactive&format=compact" +kuri-skill snap # 6. Click a result -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/action?action=click&ref=e3" +kuri-skill action click e3 # 7. Verify new page -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/page/info" +kuri-skill page-info # 8. Screenshot for evidence -curl -s -H "X-Kuri-Session: $SESSION" \ - "http://127.0.0.1:8080/screenshot" -o result.png +kuri-skill screenshot ``` --- ## Quick Parameter Reference -### Common query parameters - -| Parameter | Type | Applies to | Description | -|-----------|------|-----------|-------------| -| `tab_id` | string | Most endpoints | Target a specific tab | -| `ref` | string | `/action` | Element reference from snapshot | -| `action` | string | `/action` | `click`, `type`, `fill`, `select`, `scroll` | -| `value` | string | type / fill / select | Text or option value | -| `direction` | string | scroll | `up`, `down`, `left`, `right` | -| `amount` | number | scroll | Scroll distance (lines or pixels) | -| `button` | string | click | `left`, `right`, `middle` | -| `clickCount` | number | click | `1` = single, `2` = double | -| `modifiers` | string | click | `shift`, `ctrl`, `meta` (comma-separated) | -| `expression` | string | `/evaluate` | JavaScript to execute | -| `filter` | string | `/snapshot` | `interactive`, `all` | -| `format` | string | `/snapshot` | `compact`, `json`, `verbose` | -| `session` | string | Most endpoints | `X-Kuri-Session` header override | -| `wait` | number | Most endpoints | Milliseconds to wait before response | -| `timeout` | number | Most endpoints | Operation timeout in milliseconds | -| `url` | string | `/navigate`, `/tab/new` | Target URL (URL-encoded) | - -### Full endpoint map - -| Method | Endpoint | Purpose | -|--------|----------|---------| -| GET | `/health` | Server health check | -| GET | `/tabs` | List all tabs | -| GET/POST | `/tab/new` | Create a new tab | -| GET | `/tab/close` | Close a tab | -| GET | `/tab/current` | Set or get current tab | -| GET | `/navigate` | Navigate a tab to a URL | -| GET | `/page/info` | Current page metadata (URL, title, state) | -| GET | `/page/state` | Lightweight page observation (48 tokens) | -| GET | `/screenshot` | Capture viewport screenshot | -| GET | `/snapshot` | Accessibility tree snapshot | -| GET | `/text` | Page plain text | -| GET | `/markdown` | Page as markdown | -| GET | `/links` | Extract all links | -| GET | `/action` | Perform action (click, type, fill, select, scroll) | -| GET | `/evaluate` | Execute JavaScript | -| GET | `/cookies` | List cookies with security flags | -| GET | `/audit` | Full security audit | -| GET | `/back` | Browser back | -| GET | `/forward` | Browser forward | -| GET | `/reload` | Reload page | -| GET | `/har` | Current HAR log | -| GET | `/har/start` | Start HAR recording | -| GET | `/har/stop` | Stop and return HAR | -| GET | `/headers` | Response security headers | -| GET | `/token` | Print or rotate API token | -| GET | `/discover` | Discover CDP tabs | -| POST | `/batch` | Execute multiple commands in one call | -| GET | `/element/state` | Quick element state check | -| GET | `/find-element` | Semantic locator search | -| GET | `/clipboard/read` | Read clipboard content | -| GET | `/clipboard/write` | Write to clipboard | -| GET | `/dialog/auto` | Auto-accept dialogs | -| GET | `/dialog/accept` | Accept current dialog | -| GET | `/dialog/dismiss` | Dismiss current dialog | -| GET | `/mouse/move` | Move mouse to coordinates | -| GET | `/mouse/down` | Mouse button down | -| GET | `/mouse/up` | Mouse button up | -| GET | `/mouse/wheel` | Mouse wheel scroll | -| GET | `/timezone` | Set emulated timezone | -| GET | `/locale` | Set emulated locale | -| GET | `/permissions` | Set browser permissions | -| GET | `/clear` | Clear input content | -| GET | `/selectall` | Select all in input | -| GET | `/boundingbox` | Element bounding box | -| GET | `/getattribute` | Get element attribute | -| GET | `/inputvalue` | Get input value | -| GET | `/evalhandle` | Evaluate and return JS handle | -| GET | `/recording/start` | Start action recording | -| GET | `/recording/stop` | Stop and export recording | -| GET | `/recording/export` | Export recording as batch JSON | -| GET | `/vitals` | Core Web Vitals (LCP, CLS, FID, TTFB, FCP) | -| GET | `/pushstate` | Simulate pushState navigation | -| GET | `/bringtofront` | Bring tab to front | -| GET | `/frame` | Switch iframe context | -| GET | `/mainframe` | Switch to main frame | -| GET | `/addstyle` | Inject CSS into page | -| GET | `/expose` | Expose Node.js function to page | -| GET | `/setcontent` | Set page HTML content | -| GET | `/request/detail` | Request detail by ID | -| GET | `/response/body` | Response body by request ID | -| GET | `/download` | Download a file | -| GET | `/diff/url` | Compare two URLs side by side | -| GET | `/cache/set` | Cache a value | -| GET | `/cache/get` | Get cached value | -| GET | `/cache/clear` | Clear cache | -| GET | `/cache/list` | List cache keys | -| GET | `/screenshot/som` | Set-of-Marks screenshot with numbered overlays | -| GET | `/snapshot/changes` | Smart diff โ€” only changed lines since last snapshot | - -See the main README for a complete, always-up-to-date endpoint list. +| Parameter | Applies to | Description | +|-----------|-----------|-------------| +| `ref` | action | Element reference from snapshot | +| `action` | action | `click`, `type`, `fill`, `select`, `scroll` | +| `value` | type / fill / select | Text or option value | +| `direction` | scroll | `up`, `down`, `left`, `right` | +| `amount` | scroll | Scroll distance (lines or pixels) | +| `button` | click | `left`, `right`, `middle` | +| `clickCount` | click | `1` = single, `2` = double | +| `modifiers` | click | `shift`, `ctrl`, `meta` | +| `expression` | evaluate | JavaScript to execute | +| `tab_id` | Most commands | Target a specific tab | + +## Full Endpoint Map + +The `kuri-skill` CLI wraps all major Kuri HTTP endpoints. For the complete +list of ~80+ endpoints, see the main Kuri README. + +| Endpoint | `kuri-skill` | Purpose | +|----------|-------------|---------| +| `/health` | `health` | Server health check | +| `/tabs` | `tabs` | List all tabs | +| `/tab/new` | `tab-new` | Create a new tab | +| `/tab/close` | `action close-tab` | Close a tab | +| `/tab/current` | โ€” (use `KURI_SESSION`) | Set current tab | +| `/navigate` | `navigate` | Navigate to URL | +| `/page/info` | `page-info` | Page metadata | +| `/page/state` | โ€” | Lightweight observation | +| `/screenshot` | `screenshot` | Capture screenshot | +| `/snapshot` | `snap` | Accessibility tree | +| `/text` | `text` | Page plain text | +| `/markdown` | `markdown` | Page as markdown | +| `/links` | `links` | Extract links | +| `/action` | `action ` | Click, type, fill, select, scroll | +| `/evaluate` | `action evaluate` | Execute JavaScript | +| `/cookies` | `action cookies` | List cookies | +| `/audit` | `action audit` | Security audit | +| `/back` | `action back` | Browser back | +| `/forward` | `action forward` | Browser forward | +| `/reload` | `action reload` | Reload page | +| `/har` | `action har` | Current HAR log | +| `/har/start` | `action har-start` | Start HAR recording | +| `/har/stop` | `action har-stop` | Stop HAR recording | +| `/batch` | โ€” | Multi-command batch | +| `/evaluate` | `action evaluate` | JS evaluation | +| `/clipboard/*` | โ€” | Clipboard read/write | +| `/dialog/*` | โ€” | Dialog handling | +| `/mouse/*` | โ€” | Mouse control | +| `/recording/*` | โ€” | Action recording | +| `/vitals` | โ€” | Core Web Vitals | +| `/cache/*` | โ€” | Action caching | + +For the complete list see the main [Kuri README](../readme.md). diff --git a/skills/pi-kuri-plugin/.npmignore b/skills/pi-kuri-plugin/.npmignore new file mode 100644 index 0000000..7a7b33a --- /dev/null +++ b/skills/pi-kuri-plugin/.npmignore @@ -0,0 +1,2 @@ +package-lock.json +node_modules diff --git a/skills/pi-kuri-plugin/README.md b/skills/pi-kuri-plugin/README.md new file mode 100644 index 0000000..e18fcd3 --- /dev/null +++ b/skills/pi-kuri-plugin/README.md @@ -0,0 +1,144 @@ +# pi-kuri-skill + +A [Pi](https://pi.dev) skill plugin for **Kuri** browser automation โ€” navigate, +screenshot, extract page content, and interact with web pages via Kuri's HTTP API. + +This package provides: + +- **`kuri-skill` CLI** โ€” a portable Node.js wrapper around Kuri's HTTP API +- **`SKILL.md`** โ€” pi.dev agent skill with two-tier progressive disclosure +- **`references/ADVANCED.md`** โ€” on-demand reference for click, type, JS eval, etc. + +## Prerequisites + +- [Kuri](https://github.com/justrach/kuri) โ€” the Zig-native browser automation server +- Node.js 18+ + +## Install + +### As a pi.dev skill package + +```bash +pi install npm:pi-kuri-skill +``` + +### Standalone (global CLI) + +```bash +npm install -g pi-kuri-skill +``` + +Then use the `kuri-skill` command: + +```bash +kuri-skill health +kuri-skill navigate https://example.com +kuri-skill screenshot +``` + +### Project-local + +```bash +npm install pi-kuri-skill +npx kuri-skill health +``` + +### From the kuri repository + +```bash +cd skills/pi-kuri-plugin +npm install +npm link # makes `kuri-skill` available globally +``` + +## Quick Start + +1. **Start Kuri** (if not already running): + + ```bash + kuri + ``` + +2. **Check the server is alive:** + + ```bash + kuri-skill health + ``` + +3. **Navigate to a page:** + + ```bash + kuri-skill navigate https://example.com + ``` + +4. **Inspect the page:** + + ```bash + kuri-skill page-info + kuri-skill screenshot + kuri-skill text + ``` + +5. **Use interactive refs:** + + ```bash + kuri-skill snap + kuri-skill action click e1 + ``` + +## Core Commands + +| Command | Description | +|---------|-------------| +| `kuri-skill health` | Server health check | +| `kuri-skill tabs` | List all browser tabs | +| `kuri-skill tab-new [url]` | Open new tab | +| `kuri-skill navigate ` | Navigate to URL | +| `kuri-skill page-info` | Current page info | +| `kuri-skill screenshot` | Capture screenshot | +| `kuri-skill text` | Extract plain text | +| `kuri-skill markdown` | Extract as markdown | +| `kuri-skill links` | Extract all links | +| `kuri-skill snap` | Accessibility snapshot | +| `kuri-skill action <...>` | Advanced actions | +| `kuri-skill advanced` | Print advanced reference path | + +## Advanced Reference + +```bash +kuri-skill advanced +``` + +Or read `references/ADVANCED.md` directly for the full API reference covering +~80 endpoints, options, and workflow patterns. + +## Configuration + +| Variable | Default | Description | +|----------|---------|-------------| +| `KURI_BASE_URL` | `http://127.0.0.1:8080` | Kuri server URL | +| `KURI_SESSION` | `pi-kuri-skill` | Active session ID | +| `KURI_API_TOKEN` | (from `~/.kuri/api.token`) | API auth token | +| `KURI_TAB_ID` | (empty) | Default tab ID override | +| `KURI_OUTPUT` | `/tmp/kuri-*.png` | Screenshot output path | +| `KURI_PORT` | `8080` | Server port | + +## Structure + +``` +pi-kuri-plugin/ +โ”œโ”€โ”€ SKILL.md # Pi agent skill (loaded by pi) +โ”œโ”€โ”€ package.json # npm package metadata + pi.skills +โ”œโ”€โ”€ README.md # This file +โ”œโ”€โ”€ scripts/ +โ”‚ โ””โ”€โ”€ kuri.js # CLI wrapper (also the `kuri-skill` bin) +โ””โ”€โ”€ references/ + โ””โ”€โ”€ ADVANCED.md # On-demand advanced reference +``` + +## Related + +- [Kuri](https://github.com/justrach/kuri) โ€” Zig-native browser automation server +- [Pi](https://pi.dev) โ€” The coding agent harness +- [`skills/pi-kuri-skills.md`](../pi-kuri-skills.md) โ€” Canonical curl-based core skill +- [`skills/pi-kuri-advanced.md`](../pi-kuri-advanced.md) โ€” Canonical curl-based advanced reference diff --git a/skills/pi-kuri-plugin/SKILL.md b/skills/pi-kuri-plugin/SKILL.md new file mode 100644 index 0000000..3d110d4 --- /dev/null +++ b/skills/pi-kuri-plugin/SKILL.md @@ -0,0 +1,176 @@ +--- +name: pi-kuri +description: Kuri browser automation for pi.dev agents โ€” navigate, screenshot, extract content, and interact with web pages via the Kuri HTTP server. Use when an agent needs to browse the web, read pages, take screenshots, or perform browser interactions. This is the pi.dev plugin skill โ€” see skills/pi-kuri-skills.md for the canonical reference. +--- + +# Pi Kuri Plugin + +Kuri is a Zig-native browser automation server. This plugin provides a CLI +wrapper (`kuri-skill`) and on-demand reference docs for agent-driven browser +automation via Kuri's HTTP API. + +### Getting `kuri-skill` on PATH + +```bash +# From the plugin directory (when installed from npm): +npm link + +# Or run directly: +node path/to/scripts/kuri.js + +# When installed globally: +npm install -g pi-kuri-skill +``` + +This skill has two tiers: + +| Tier | Covered here | When to use | +|------|-------------|-------------| +| **Core** ๐ŸŸข | Server health check, start, navigate, screenshot, page info, text & markdown extraction, links, accessibility snapshots | Most tasks | +| **Advanced** ๐Ÿ”ต | Click, type, fill, select, scroll, JS eval, cookies, audit, HAR, session mgmt | Run `kuri-skill advanced` or read `skills/pi-kuri-advanced.md` | + +> **Direct HTTP alternative:** If you don't have Node.js, all operations work +> via direct HTTP calls โ€” see `skills/pi-kuri-skills.md` for details. + +--- + +## 1. Setup โ€” Validate & Start + +### Check if Kuri is running + +```bash +kuri-skill health +``` + +A healthy server returns `{"ok":true}` with the server version. + +If the health check fails โ€” connection refused or timeout โ€” the server is not running. + +### Start Kuri + +If you built from source: + +```bash +# From the kuri repo root +zig build +./zig-out/bin/kuri +``` + +If you installed via the install script: + +```bash +kuri +``` + +Kuri starts a Chrome instance (managing it automatically) and listens on +`http://127.0.0.1:8080` by default. Customize with `HOST`, `PORT`, `CDP_URL` +(see the configuration table in the README). + +### List available tabs + +```bash +kuri-skill tabs +``` + +Returns the active session's tabs with their IDs, URLs, and titles. + +--- + +## 2. Core Operations + +All examples use the `kuri-skill` CLI. + +### Create a tab and navigate + +```bash +kuri-skill tab-new https://example.com +kuri-skill navigate https://example.com +``` + +### Get page info + +```bash +kuri-skill page-info +``` + +Returns `url`, `title`, and page `readyState`. Always call this after navigation +to confirm the page loaded before taking further actions. + +### Screenshot + +```bash +kuri-skill screenshot +``` + +Saves a PNG. Customize the output path with `KURI_OUTPUT`. + +### Content extraction + +```bash +kuri-skill text +kuri-skill markdown +kuri-skill links +``` + +### Accessibility snapshot (interactive refs) + +```bash +kuri-skill snap +``` + +Returns a compact text-tree of interactive elements with refs like `e0`, `e1`. +These refs are used as targets for advanced actions (click, type, fill, etc.). + +**Re-snap after any navigation or DOM change** โ€” refs are snapshot-local and +become stale once the page updates. + +--- + +## 3. Advanced Operations + +For click, type, fill, select, scroll, JavaScript evaluation, cookies, security +audits, HAR recording, and session management: + +```bash +kuri-skill advanced # Print the path to the advanced reference +``` + +Or read the reference directly: + +```bash +read skills/pi-kuri-plugin/references/ADVANCED.md +``` + +The canonical advanced reference is at `skills/pi-kuri-advanced.md`. + +**What's available at a glance:** + +| Category | Capabilities | +|----------|-------------| +| ๐Ÿ–ฑ๏ธ Mouse | click, right-click, double-click, hover | +| โŒจ๏ธ Keyboard | type text, fill input, select option | +| ๐Ÿ“œ Scroll | up, down, left, right, pixel-amount | +| ๐Ÿง  JS Eval | arbitrary JavaScript, return values | +| ๐Ÿช Cookies | list, inspect security flags | +| ๐Ÿ”’ Security | audit headers, cookies, HTTPS, mixed content | +| ๐Ÿ“ก HAR | HTTP archive recording (start, stop, export) | +| ๐Ÿ“‘ Tabs & Sessions | multi-tab, multi-session orchestration | + +--- + +## Tips + +- **Prefer sessions** โ€” set `KURI_SESSION` env var to group tabs without + repeating `tab_id` on every call. +- **Call `page-info` before acting** โ€” confirm the right page is loaded. +- **Use `KURI_ALLOW_PRIVATE=1`** โ€” if you need to reach localhost or private + IPs during development, set this before starting the Kuri server. + +## Configuration + +| Variable | Default | Description | +|----------|---------|-------------| +| `KURI_BASE_URL` | `http://127.0.0.1:8080` | Kuri server URL | +| `KURI_SESSION` | `pi-kuri-skill` | Active session ID | +| `KURI_API_TOKEN` | (from `~/.kuri/api.token`) | API auth token | +| `KURI_OUTPUT` | `/tmp/kuri-*.png` | Screenshot output path | diff --git a/skills/pi-kuri-plugin/package.json b/skills/pi-kuri-plugin/package.json new file mode 100644 index 0000000..38fc72b --- /dev/null +++ b/skills/pi-kuri-plugin/package.json @@ -0,0 +1,22 @@ +{ + "name": "pi-kuri-skill", + "version": "1.0.0", + "description": "Pi skill for Kuri browser automation โ€” navigate, screenshot, interact, extract. Installable pi.dev plugin for the Kuri Zig-native browser automation server.", + "keywords": ["pi-package", "agent-skills", "kuri", "browser-automation"], + "type": "module", + "bin": { + "kuri-skill": "./scripts/kuri.js" + }, + "pi": { + "skills": ["."] + }, + "scripts": { + "test": "node scripts/kuri.js health", + "start": "node scripts/kuri.js" + }, + "dependencies": {}, + "license": "MIT", + "engines": { + "node": ">=18" + } +} diff --git a/skills/pi-kuri-plugin/references/ADVANCED.md b/skills/pi-kuri-plugin/references/ADVANCED.md new file mode 100644 index 0000000..8d9ff81 --- /dev/null +++ b/skills/pi-kuri-plugin/references/ADVANCED.md @@ -0,0 +1,354 @@ +--- +name: pi-kuri-advanced +description: Advanced Kuri browser automation for pi.dev agents โ€” click, type, fill, select, scroll, JavaScript evaluation, cookies, security audits, session management, HAR recording, and the experimental kuri-browser CLI. Load this on demand when the core skill (`skills/pi-kuri-skills.md`) is not enough. +--- + +# Pi Kuri Advanced Reference + +This file covers operations beyond the core skill (`skills/pi-kuri-skills.md`). +Load it when you need interactive actions, JavaScript evaluation, cookies, +security audits, session management, or experimental features. + +All commands use the `kuri-skill` CLI from `skills/pi-kuri-plugin/`. If +`kuri-skill` is not on your PATH, run commands via: + +```bash +node skills/pi-kuri-plugin/scripts/kuri.js +``` + +Set `KURI_SESSION` to group tabs without repeating `tab_id` on every call: + +```bash +export KURI_SESSION=my-session +``` + +--- + +## ๐Ÿ–ฑ๏ธ Mouse Actions + +### Click an element + +```bash +kuri-skill action click e1 +kuri-skill action click ref=e1 tab_id=TABID123 + +# Right-click +kuri-skill action click ref=e1 button=right + +# Double-click +kuri-skill action click ref=e1 clickCount=2 +``` + +### Hover + +Use the snapshot to detect hover-triggered UI, then evaluate JS for precise +hover scenarios: + +```bash +kuri-skill action evaluate \ + expression="document.querySelector('button').dispatchEvent(new MouseEvent('mouseover',{bubbles:true}))" +``` + +--- + +## โŒจ๏ธ Keyboard Actions + +### Type text into an input + +```bash +kuri-skill action type e2 "hello world" +kuri-skill action type ref=e2 value="hello world" +``` + +### Fill input value + +```bash +kuri-skill action fill e2 "user@example.com" +kuri-skill action fill ref=e2 value="user@example.com" +``` + +### Select a dropdown option + +```bash +kuri-skill action select e3 "option-value" +kuri-skill action select ref=e3 value="option-value" +``` + +--- + +## ๐Ÿ“œ Scrolling + +```bash +# Scroll down (default) +kuri-skill action scroll + +# Scroll up +kuri-skill action scroll direction=up amount=10 + +# Scroll right +kuri-skill action scroll direction=right amount=3 + +# Scroll by pixel amount +kuri-skill action scroll direction=down amount=500 +``` + +--- + +## ๐Ÿง  JavaScript Evaluation + +```bash +# Get page title +kuri-skill action evaluate expression="document.title" + +# Read heading text +kuri-skill action evaluate \ + expression="document.querySelector('h1').textContent" + +# Extract dynamic state +kuri-skill action evaluate \ + expression="JSON.stringify(window.__INITIAL_STATE__)" + +# Get page metrics +kuri-skill action evaluate \ + expression="document.body.scrollHeight" + +# Check element visibility +kuri-skill action evaluate \ + expression="document.querySelector('.spinner').offsetParent!==null" +``` + +Returns the evaluated result as a string, number, boolean, or JSON value. + +--- + +## ๐Ÿช Cookies + +```bash +kuri-skill action cookies +``` + +Returns an array of cookies with: `name`, `value`, `domain`, `path`, `secure`, +`httpOnly`, `sameSite`, `expires`. + +--- + +## ๐Ÿ”’ Security Audit + +```bash +kuri-skill action audit +``` + +Checks performed: +- Security response headers: HSTS, CSP, X-Frame-Options, X-Content-Type-Options +- Cookie security: Secure flag, HttpOnly flag, SameSite attribute +- HTTPS enforcement +- Mixed content warnings + +--- + +## ๐Ÿ“ก HAR Recording (HTTP Archive) + +Record network activity for API discovery or debugging: + +```bash +# Start recording +kuri-skill action har-start + +# Perform actions that make network requests... + +# Stop and retrieve the HAR log +kuri-skill action har-stop + +# Get current HAR log without stopping +kuri-skill action har +``` + +Useful for reverse-engineering API calls or debugging network issues. + +--- + +## ๐Ÿ“‘ Tab & Session Management + +```bash +# List all tabs +kuri-skill tabs + +# Create a new tab +kuri-skill tab-new https://example.com + +# Close a tab +kuri-skill action close-tab TABID123 + +# Browser back +kuri-skill action back + +# Browser forward +kuri-skill action forward + +# Reload +kuri-skill action reload +``` + +### Multiple sessions + +Run independent browser sessions by setting `KURI_SESSION` to different values: + +```bash +KURI_SESSION=session-a kuri-skill tab-new https://example.com +KURI_SESSION=session-b kuri-skill tab-new https://other.com +``` + +Each session tracks its own tabs and current tab independently. + +--- + +## ๐Ÿงช Experimental โ€” kuri-browser + +The `kuri-browser/` subdirectory is an **experimental** Zig-native browser +runtime with its own rendering engine (separate build, not wired into Kuri's +root build). Use for evaluation and development only. + +```bash +cd kuri-browser +zig build run -- render https://news.ycombinator.com --selector ".titleline a" --dump text +zig build run -- render https://todomvc.com/examples/react/dist/ --js --wait-eval "..." +zig build run -- bench --offline +zig build run -- serve-cdp --port 9333 +``` + +Screenshots fall back to the main Kuri/CDP renderer โ€” start the normal Kuri +server first, then: + +```bash +cd kuri-browser +zig build run -- screenshot https://example.com --out example.jpg \ + --compress --kuri-base http://127.0.0.1:8080 +``` + +--- + +## Common Workflow Patterns + +### Pattern 1: Quick screenshot verification + +```bash +export KURI_SESSION=verify + +kuri-skill tab-new https://example.com +kuri-skill page-info +kuri-skill screenshot # set KURI_OUTPUT to customize path +``` + +### Pattern 2: Fill form and submit + +```bash +export KURI_SESSION=login + +kuri-skill tab-new https://example.com/login +kuri-skill snap # Get refs + +kuri-skill action type e0 "user@example.com" +kuri-skill action type e1 "password123" +kuri-skill action click e2 # Submit button + +kuri-skill page-info # Check result +``` + +### Pattern 3: Extract dynamic content + +```bash +export KURI_SESSION=extract + +kuri-skill tab-new https://example.com/app +kuri-skill action evaluate expression="JSON.stringify(window.__DATA__)" +kuri-skill text # Also get rendered text +``` + +### Pattern 4: Multi-step interaction loop + +```bash +export KURI_SESSION=search + +# 1. Open page +kuri-skill tab-new https://example.com/search + +# 2. Get snapshot for refs +kuri-skill snap + +# 3. Type search query +kuri-skill action type e0 "query text" + +# 4. Check for results via JS +kuri-skill action evaluate \ + expression="document.querySelectorAll('.result').length" + +# 5. Re-snap after DOM change +kuri-skill snap + +# 6. Click a result +kuri-skill action click e3 + +# 7. Verify new page +kuri-skill page-info + +# 8. Screenshot for evidence +kuri-skill screenshot +``` + +--- + +## Quick Parameter Reference + +| Parameter | Applies to | Description | +|-----------|-----------|-------------| +| `ref` | action | Element reference from snapshot | +| `action` | action | `click`, `type`, `fill`, `select`, `scroll` | +| `value` | type / fill / select | Text or option value | +| `direction` | scroll | `up`, `down`, `left`, `right` | +| `amount` | scroll | Scroll distance (lines or pixels) | +| `button` | click | `left`, `right`, `middle` | +| `clickCount` | click | `1` = single, `2` = double | +| `modifiers` | click | `shift`, `ctrl`, `meta` | +| `expression` | evaluate | JavaScript to execute | +| `tab_id` | Most commands | Target a specific tab | + +## Full Endpoint Map + +The `kuri-skill` CLI wraps all major Kuri HTTP endpoints. For the complete +list of ~80+ endpoints, see the main Kuri README. + +| Endpoint | `kuri-skill` | Purpose | +|----------|-------------|---------| +| `/health` | `health` | Server health check | +| `/tabs` | `tabs` | List all tabs | +| `/tab/new` | `tab-new` | Create a new tab | +| `/tab/close` | `action close-tab` | Close a tab | +| `/tab/current` | โ€” (use `KURI_SESSION`) | Set current tab | +| `/navigate` | `navigate` | Navigate to URL | +| `/page/info` | `page-info` | Page metadata | +| `/page/state` | โ€” | Lightweight observation | +| `/screenshot` | `screenshot` | Capture screenshot | +| `/snapshot` | `snap` | Accessibility tree | +| `/text` | `text` | Page plain text | +| `/markdown` | `markdown` | Page as markdown | +| `/links` | `links` | Extract links | +| `/action` | `action ` | Click, type, fill, select, scroll | +| `/evaluate` | `action evaluate` | Execute JavaScript | +| `/cookies` | `action cookies` | List cookies | +| `/audit` | `action audit` | Security audit | +| `/back` | `action back` | Browser back | +| `/forward` | `action forward` | Browser forward | +| `/reload` | `action reload` | Reload page | +| `/har` | `action har` | Current HAR log | +| `/har/start` | `action har-start` | Start HAR recording | +| `/har/stop` | `action har-stop` | Stop HAR recording | +| `/batch` | โ€” | Multi-command batch | +| `/evaluate` | `action evaluate` | JS evaluation | +| `/clipboard/*` | โ€” | Clipboard read/write | +| `/dialog/*` | โ€” | Dialog handling | +| `/mouse/*` | โ€” | Mouse control | +| `/recording/*` | โ€” | Action recording | +| `/vitals` | โ€” | Core Web Vitals | +| `/cache/*` | โ€” | Action caching | + +For the complete list see the main [Kuri README](../readme.md). diff --git a/skills/pi-kuri-plugin/scripts/kuri.js b/skills/pi-kuri-plugin/scripts/kuri.js new file mode 100755 index 0000000..1d4b8ea --- /dev/null +++ b/skills/pi-kuri-plugin/scripts/kuri.js @@ -0,0 +1,622 @@ +#!/usr/bin/env node +/** + * kuri-skill โ€” Pi skill CLI for Kuri browser automation. + * + * Available as a standalone command after npm install -g: + * kuri-skill health + * kuri-skill navigate + * ... + * + * Or run directly: + * node scripts/kuri.js health + * + * Core operations are always available directly. + * Advanced operations (click, type, fill, audit, etc.) are documented + * in references/ADVANCED.md and invoked via 'action '. + * + * Usage: + * kuri-skill health + * kuri-skill tabs + * kuri-skill navigate [tab_id] + * kuri-skill tab-new [url] + * kuri-skill screenshot [tab_id] [--output path] + * kuri-skill page-info [tab_id] + * kuri-skill text [tab_id] + * kuri-skill markdown [tab_id] + * kuri-skill links [tab_id] + * kuri-skill snap [tab_id] + * kuri-skill action [args...] + * kuri-skill advanced # Print reference to ADVANCED.md + */ + +import { readFileSync, writeFileSync, existsSync, mkdirSync } from "node:fs"; +import { createWriteStream } from "node:fs"; +import { get } from "node:https"; +import { homedir } from "node:os"; +import { resolve, dirname } from "node:path"; +import { fileURLToPath } from "node:url"; +import { execSync } from "node:child_process"; + +// โ”€โ”€ Configuration โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +const __dirname = dirname(fileURLToPath(import.meta.url)); +const SKILL_ROOT = resolve(__dirname, ".."); + +function kuriBaseUrl() { + return process.env.KURI_BASE_URL || "http://127.0.0.1:8080"; +} + +function kuriApiToken() { + if (process.env.KURI_API_TOKEN) return process.env.KURI_API_TOKEN; + if (process.env.KURI_SECRET) return process.env.KURI_SECRET; + try { + return readFileSync(`${homedir()}/.kuri/api.token`, "utf-8").trim(); + } catch { + return ""; + } +} + +function defaultSession() { + return process.env.KURI_SESSION || "pi-kuri-skill"; +} + +function sessionTabId() { + return process.env.KURI_TAB_ID || ""; +} + +function headers(session) { + const h = { + "X-Kuri-Session": session || defaultSession(), + Accept: "application/json", + }; + const token = kuriApiToken(); + if (token) h["Authorization"] = `Bearer ${token}`; + return h; +} + +// โ”€โ”€ Error recovery โ€” detect if Kuri server is not running โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +function isKuriInstalled() { + try { + execSync("which kuri", { stdio: "ignore" }); + return true; + } catch { + return false; + } +} + +function isBuiltLocally() { + return existsSync(resolve(SKILL_ROOT, "..", "..", "zig-out", "bin", "kuri")); +} + +function platform() { + const os = process.platform; + if (os === "darwin") return "macos"; + if (os === "linux") return "linux"; + return os; +} + +/** Print structured, LLM-parsable recovery hints when Kuri server is unreachable. */ +function printKuriNotRunning(originalError) { + const hints = { + status: "kuri_not_running", + message: "Kuri server is not reachable. The server must be running on " + + `${kuriBaseUrl()} before any browser operations work.`, + error: originalError, + platform: platform(), + kuri_on_path: isKuriInstalled(), + built_in_repo: isBuiltLocally(), + suggestions: [] + }; + + if (isKuriInstalled()) { + hints.suggestions.push({ + action: "start_kuri", + command: "kuri", + detail: "Kuri is installed on PATH. Start it with the 'kuri' command." + }); + } + + if (isBuiltLocally()) { + hints.suggestions.push({ + action: "start_kuri_local", + command: "./zig-out/bin/kuri", + cwd: resolve(SKILL_ROOT, "..", ".."), + detail: "A local build exists in the kuri repo. Start it from the repo root." + }); + } + + if (!isKuriInstalled() && !isBuiltLocally()) { + const installScript = + "curl -fsSL https://raw.githubusercontent.com/justrach/kuri/main/install.sh | sh"; + hints.suggestions.push({ + action: "install_kuri", + install_url: "https://github.com/justrach/kuri", + install_script: installScript, + detail: "Kuri is not installed. Use the one-line install script, or build from source." + }); + hints.suggestions.push({ + action: "build_from_source", + clone_url: "https://github.com/justrach/kuri.git", + build_commands: ["git clone https://github.com/justrach/kuri.git", "cd kuri", "zig build", "./zig-out/bin/kuri"], + requires_zig: true, + detail: "Clone the kuri repo and build with Zig 0.16." + }); + } + + hints.suggestions.push({ + action: "check_port_or_url", + detail: `Verify the server URL. The default is http://127.0.0.1:8080. ` + + `Override with KURI_BASE_URL env var.` + }); + + console.log(JSON.stringify(hints, null, 2)); +} + +// โ”€โ”€ HTTP helpers โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +async function kuriFetch(path, query = {}, method = "GET", body = null) { + const base = kuriBaseUrl().replace(/\/+$/, ""); + const params = new URLSearchParams(query).toString(); + const url = `${base}${path}${params ? `?${params}` : ""}`; + const opts = { + method, + headers: headers(query.session || ""), + signal: AbortSignal.timeout(30_000), + }; + if (body) { + opts.headers["Content-Type"] = "application/json"; + opts.body = JSON.stringify(body); + } + try { + const resp = await fetch(url, opts); + const ct = resp.headers.get("content-type") || ""; + if (ct.includes("image/png") || ct.includes("image/")) { + const buf = await resp.arrayBuffer(); + return { _binary: true, data: Buffer.from(buf), contentType: ct }; + } + if (!resp.ok) { + const text = await resp.text().catch(() => ""); + throw new Error(`Kuri ${path}: ${resp.status} ${resp.statusText} โ€” ${text.slice(0, 300)}`); + } + return resp.json(); + } catch (err) { + // Distinguish connection refused (server not running) from other errors + if (err?.cause?.code === "ECONNREFUSED" || err?.message?.includes("ECONNREFUSED") || err?.message?.includes("fetch failed")) { + const nfe = new Error("KURI_NOT_RUNNING"); + nfe.code = "KURI_NOT_RUNNING"; + nfe.originalMessage = err.message; + throw nfe; + } + throw err; + } +} + +// โ”€โ”€ Output helpers โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +function printJson(data) { + console.log(JSON.stringify(data, null, 2)); +} + +function printText(label, data) { + if (typeof data === "string") { + console.log(data); + return; + } + if (data._binary) { + console.log(`[binary: ${data.data.length} bytes, ${data.contentType}]`); + return; + } + // Navigate through common response shapes + let result = data; + if (data && typeof data === "object") { + if (data.result && typeof data.result === "object") { + const r = data.result; + if (r.result && r.result.value !== undefined) result = r.result.value; + else if (r.value !== undefined) result = r.value; + else if (r.text !== undefined) result = r.text; + else if (r.data !== undefined) result = r.data; + else if (r.markdown !== undefined) result = r.markdown; + else result = r; + } else if (data.value !== undefined) { + result = data.value; + } else if (data.text !== undefined) { + result = data.text; + } else if (data.markdown !== undefined) { + result = data.markdown; + } + } + if (typeof result === "string") { + console.log(result); + } else { + console.log(JSON.stringify(result, null, 2)); + } +} + +// โ”€โ”€ Core operations โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +async function cmdHealth() { + const data = await kuriFetch("/health"); + printJson(data); +} + +async function cmdTabs() { + const data = await kuriFetch("/tabs"); + printJson(data); +} + +async function cmdTabNew(url) { + const params = {}; + if (url) params.url = encodeURI(url); + const data = await kuriFetch("/tab/new", params); + const tabId = data?.result?.tabId || data?.result?.id || data?.id || ""; + if (tabId) { + // Store the tab ID for convenience + console.log(`Tab created: ${tabId}`); + // Also print full response + printJson(data); + } else { + printJson(data); + } +} + +async function cmdNavigate(url, tabId) { + if (!url) { + console.error("Usage: kuri.js navigate [tab_id]"); + process.exit(1); + } + if (!tabId) { + // Try to find the current tab + try { + const tabs = await kuriFetch("/tabs"); + const list = Array.isArray(tabs) ? tabs : (tabs.tabs || tabs.result || []); + const current = list.find((t) => t.current); + if (current) tabId = current.id; + } catch { /* ignore */ } + } + if (!tabId) { + console.error("No tab available. Open one first: kuri.js tab-new "); + process.exit(1); + } + const params = { url, tab_id: tabId }; + const data = await kuriFetch("/navigate", params); + printText("Navigated", data); +} + +async function resolveTabId(requested) { + if (requested) return requested; + if (sessionTabId()) return sessionTabId(); + try { + const tabs = await kuriFetch("/tabs"); + const list = Array.isArray(tabs) ? tabs : (tabs.tabs || tabs.result || []); + const current = list.find((t) => t.current); + if (current) return current.id; + } catch { /* ignore */ } + return ""; +} + +async function cmdPageInfo(tabId) { + const params = {}; + if (tabId) params.tab_id = tabId; + // Kuri v0.4.1+ may not have /page-info; fall back to /tabs filtered by tab + try { + const data = await kuriFetch("/page-info", params); + printJson(data); + } catch (err) { + // Fallback: list tabs and find the requested one + const tabs = await kuriFetch("/tabs"); + const list = Array.isArray(tabs) ? tabs : (tabs.tabs || tabs.result || []); + if (tabId) { + const tab = list.find((t) => t.id === tabId); + if (tab) { + console.log(JSON.stringify({ url: tab.url, title: tab.title, current: tab.current || false }, null, 2)); + return; + } + } + // Show current tab info + const current = list.find((t) => t.current); + if (current) { + console.log(JSON.stringify({ url: current.url, title: current.title, tab_id: current.id }, null, 2)); + } else { + printJson(tabs); + } + } +} + +async function cmdScreenshot(tabId, outputPath) { + tabId = await resolveTabId(tabId); + const params = {}; + if (tabId) params.tab_id = tabId; + const data = await kuriFetch("/screenshot", params); + + if (data._binary) { + const path = outputPath || `/tmp/kuri-screenshot-${Date.now()}.png`; + writeFileSync(path, data.data); + console.log(`Screenshot saved: ${path} (${data.data.length} bytes)`); + return; + } + + // Handle JSON-wrapped base64 + let raw = data; + if (data.result?.data) raw = data.result.data; + else if (data.data) raw = data.data; + else if (typeof data === "string") raw = data; + + if (typeof raw === "string") { + // Strip data URI prefix + if (raw.includes(",")) raw = raw.split(",")[1]; + const buf = Buffer.from(raw, "base64"); + const path = outputPath || `/tmp/kuri-screenshot-${Date.now()}.png`; + writeFileSync(path, buf); + console.log(`Screenshot saved: ${path} (${buf.length} bytes)`); + } else { + printJson(data); + } +} + +async function cmdText(tabId) { + tabId = await resolveTabId(tabId); + const params = {}; + if (tabId) params.tab_id = tabId; + const data = await kuriFetch("/text", params); + printText("Text", data); +} + +async function cmdMarkdown(tabId) { + tabId = await resolveTabId(tabId); + const params = {}; + if (tabId) params.tab_id = tabId; + const data = await kuriFetch("/markdown", params); + printText("Markdown", data); +} + +async function cmdLinks(tabId) { + tabId = await resolveTabId(tabId); + const params = {}; + if (tabId) params.tab_id = tabId; + const data = await kuriFetch("/links", params); + printJson(data); +} + +async function cmdSnap(tabId) { + tabId = await resolveTabId(tabId); + const params = {}; + if (tabId) params.tab_id = tabId; + try { + const data = await kuriFetch("/snapshot", { ...params, filter: "interactive", format: "compact" }); + printText("Accessibility snapshot", data); + } catch { + // Fallback to full snapshot + const data = await kuriFetch("/snapshot", params); + printText("Accessibility snapshot", data); + } +} + +// โ”€โ”€ Advanced operation dispatcher โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +async function cmdAction(action, args) { + const tabId = args.tab_id || sessionTabId(); + const params = { ...args }; + if (tabId) params.tab_id = tabId; + + switch (action) { + // Navigation + case "back": { + const data = await kuriFetch("/back", params); + printText("Back", data); + break; + } + case "forward": { + const data = await kuriFetch("/forward", params); + printText("Forward", data); + break; + } + case "reload": { + const data = await kuriFetch("/reload", params); + printText("Reload", data); + break; + } + + // Interaction + case "click": { + if (!args.ref) { console.error("Usage: click [tab_id]"); process.exit(1); } + const data = await kuriFetch("/action", { ...params, action: "click", ref: args.ref }); + printText("Click", data); + break; + } + case "type": { + if (!args.ref || !args.value) { console.error("Usage: type [tab_id]"); process.exit(1); } + const data = await kuriFetch("/action", { ...params, action: "type", ref: args.ref, value: args.value }); + printText("Type", data); + break; + } + case "fill": { + if (!args.ref || !args.value) { console.error("Usage: fill [tab_id]"); process.exit(1); } + const data = await kuriFetch("/action", { ...params, action: "fill", ref: args.ref, value: args.value }); + printText("Fill", data); + break; + } + case "select": { + if (!args.ref || !args.value) { console.error("Usage: select [tab_id]"); process.exit(1); } + const data = await kuriFetch("/action", { ...params, action: "select", ref: args.ref, value: args.value }); + printText("Select", data); + break; + } + case "scroll": { + const dir = args.direction || "down"; + const amount = args.amount || ""; + const data = await kuriFetch("/action", { ...params, action: "scroll", direction: dir, ...(amount ? { amount } : {}) }); + printText("Scroll", data); + break; + } + + // Content + case "evaluate": { + if (!args.expression) { console.error("Usage: evaluate [tab_id]"); process.exit(1); } + const data = await kuriFetch("/evaluate", { ...params, expression: args.expression }); + printText("Evaluate", data); + break; + } + + // Cookies & Security + case "cookies": { + const data = await kuriFetch("/cookies", params); + printJson(data); + break; + } + case "audit": { + const data = await kuriFetch("/audit", params); + printJson(data); + break; + } + + // Tab management + case "close-tab": { + if (!args.tab_id && !tabId) { console.error("Usage: close-tab "); process.exit(1); } + const data = await kuriFetch("/tab/close", { tab_id: args.tab_id || tabId }); + printText("Close tab", data); + break; + } + + // HAR recording + case "har": { + const data = await kuriFetch("/har", params); + printJson(data); + break; + } + case "har-start": { + const data = await kuriFetch("/har/start", params); + printJson(data); + break; + } + case "har-stop": { + const data = await kuriFetch("/har/stop", params); + printJson(data); + break; + } + + default: + console.error(`Unknown action: ${action}`); + console.error("See references/ADVANCED.md for all available actions."); + process.exit(1); + } +} + +function cmdAdvanced() { + const advPath = resolve(SKILL_ROOT, "references", "ADVANCED.md"); + console.log(`\n ๐Ÿ“– Full Kuri API reference: ${advPath}`); + console.log(" Read that file with the 'read' tool for all ~100 advanced options.\n"); +} + +// โ”€โ”€ CLI routing โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +async function main() { + const cmd = process.argv[2]; + const args = process.argv.slice(3); + + if (!cmd || cmd === "--help" || cmd === "-h") { + console.log(` +Kuri Browser Automation โ€” Pi Skill CLI + +Usage: + kuri-skill [args...] + +Core Commands: + health Check if Kuri server is running + tabs List all browser tabs + tab-new [url] Open a new tab (optionally navigate to URL) + navigate [tab_id] Navigate tab to URL + page-info [tab_id] Get current page URL, title, ready state + screenshot [tab_id] Capture page screenshot + text [tab_id] Extract page text content + markdown [tab_id] Extract page as markdown + links [tab_id] Extract all links + snap [tab_id] Get accessibility snapshot (interactive refs) + +Advanced (see references/ADVANCED.md for full docs): + action click [tab_id] + action type [tab_id] + action fill [tab_id] + action select [tab_id] + action scroll [direction] [amount] [tab_id] + action evaluate [tab_id] + action cookies [tab_id] + action audit [tab_id] + action har|har-start|har-stop [tab_id] + action back|forward|reload [tab_id] + action close-tab + +Utilities: + advanced Print path to full API reference +`); + return; + } + + try { + switch (cmd) { + case "health": + await cmdHealth(); + break; + case "tabs": + await cmdTabs(); + break; + case "tab-new": + await cmdTabNew(args[0]); + break; + case "navigate": + await cmdNavigate(args[0], args[1]); + break; + case "page-info": + await cmdPageInfo(args[0]); + break; + case "screenshot": + await cmdScreenshot(args[0], process.env.KURI_OUTPUT); + break; + case "text": + await cmdText(args[0]); + break; + case "markdown": + await cmdMarkdown(args[0]); + break; + case "links": + await cmdLinks(args[0]); + break; + case "snap": + await cmdSnap(args[0]); + break; + case "action": + // Parse remaining args: action [key=value...] + const sub = args[0]; + const actionArgs = {}; + for (let i = 1; i < args.length; i++) { + if (args[i].includes("=")) { + const [k, ...v] = args[i].split("="); + actionArgs[k] = v.join("="); + } else if (!actionArgs.ref && !["back", "forward", "reload"].includes(sub)) { + // Positional args fill in as needed + if (!actionArgs.ref) actionArgs.ref = args[i]; + else if (!actionArgs.value) actionArgs.value = args[i]; + else if (!actionArgs.direction) actionArgs.direction = args[i]; + } + } + await cmdAction(sub, actionArgs); + break; + case "advanced": + cmdAdvanced(); + break; + default: + console.error(`Unknown command: ${cmd}`); + console.error("Run 'node scripts/kuri.js --help' for usage."); + process.exit(1); + } + } catch (err) { + if (err?.code === "KURI_NOT_RUNNING") { + printKuriNotRunning(err.originalMessage || err.message); + } else { + console.error(`Error: ${err.message}`); + } + process.exit(1); + } +} + +main(); diff --git a/skills/pi-kuri-skills.md b/skills/pi-kuri-skills.md index e58516b..948e5b9 100644 --- a/skills/pi-kuri-skills.md +++ b/skills/pi-kuri-skills.md @@ -1,19 +1,30 @@ --- name: pi-kuri-skills -description: Kuri browser automation for pi.dev agents โ€” navigate, screenshot, extract content, and interact with web pages via the Kuri HTTP server. Use when an agent needs to browse the web, read pages, take screenshots, or perform browser interactions. This is the pi.dev core skill โ€” read skills/pi-kuri-advanced.md for click, type, fill, scroll, JavaScript evaluation, cookies, security audits, and more. +description: Kuri browser automation for pi.dev agents โ€” navigate, screenshot, extract content, and interact with web pages via the Kuri HTTP server. Use when an agent needs to browse the web, read pages, take screenshots, or perform browser interactions. Commands use the `kuri-skill` CLI from `skills/pi-kuri-plugin/`. For advanced operations (click, type, scroll, etc.) read `skills/pi-kuri-advanced.md`. --- # Pi Kuri Skills -Kuri is a Zig-native browser automation server. Use its HTTP API for agent-driven -browser automation โ€” no Node.js, no Playwright, no Python dependencies. +Kuri is a Zig-native browser automation server. This skill uses the +`kuri-skill` CLI wrapper from `skills/pi-kuri-plugin/` to drive Kuri's +HTTP API โ€” no curl, no manual HTTP calls. + +If `kuri-skill` is not available globally, install it: + +```bash +cd skills/pi-kuri-plugin +npm install +npm link # makes `kuri-skill` available on PATH +# or +node scripts/kuri.js +``` This skill has two tiers: | Tier | Covered here | When to use | |------|-------------|-------------| -| **Core** ๐ŸŸข | Server health check, start/stop, navigate, screenshot, page info, text & markdown extraction, links, accessibility snapshots | Most tasks | -| **Advanced** ๐Ÿ”ต | Click, type, fill, select, scroll, JS eval, cookies, audit, HAR recording, tab management, session control, experimental kuri-browser | Read `skills/pi-kuri-advanced.md` on demand when you need them | +| **Core** ๐ŸŸข | Server health check, start, navigate, screenshot, page info, text & markdown extraction, links, accessibility snapshots | Most tasks | +| **Advanced** ๐Ÿ”ต | Click, type, fill, select, scroll, JS eval, cookies, audit, HAR, session mgmt | Read `skills/pi-kuri-advanced.md` on demand | --- @@ -22,7 +33,7 @@ This skill has two tiers: ### Check if Kuri is running ```bash -curl -s http://127.0.0.1:8080/health +kuri-skill health ``` A healthy server returns `{"ok":true}` with the server version. @@ -34,12 +45,12 @@ If the health check fails โ€” connection refused or timeout โ€” the server is no If you built from source: ```bash -# From the repo root +# From the kuri repo root zig build ./zig-out/bin/kuri ``` -If you installed via the script: +If you installed via the install script: ```bash kuri @@ -52,7 +63,7 @@ Kuri starts a Chrome instance (managing it automatically) and listens on ### List available tabs ```bash -curl -s http://127.0.0.1:8080/tabs +kuri-skill tabs ``` Returns the active session's tabs with their IDs, URLs, and titles. @@ -61,22 +72,17 @@ Returns the active session's tabs with their IDs, URLs, and titles. ## 2. Core Operations -All examples use `curl` against the default server at `http://127.0.0.1:8080`. -Set `BASE=http://127.0.0.1:8080` and `SESSION=my-session` for reuse. - ### Create a tab and navigate ```bash -# Open a new tab (session-scoped with X-Kuri-Session header) -curl -s -H "X-Kuri-Session: my-session" \ - "http://127.0.0.1:8080/tab/new?url=https%3A%2F%2Fexample.com" +kuri-skill tab-new https://example.com +kuri-skill navigate https://example.com ``` ### Get page info ```bash -curl -s -H "X-Kuri-Session: my-session" \ - "http://127.0.0.1:8080/page/info" +kuri-skill page-info ``` Returns `url`, `title`, and page `readyState`. Always call this after navigation @@ -85,36 +91,26 @@ to confirm the page loaded before taking further actions. ### Screenshot ```bash -curl -s -H "X-Kuri-Session: my-session" \ - "http://127.0.0.1:8080/screenshot" -o screenshot.png +kuri-skill screenshot ``` -Returns a PNG. Add `?format=jpeg&quality=80` for smaller files. +Saves a PNG. Customize the output path with `KURI_OUTPUT`. ### Content extraction ```bash -# Plain text -curl -s -H "X-Kuri-Session: my-session" \ - "http://127.0.0.1:8080/text" - -# Markdown -curl -s -H "X-Kuri-Session: my-session" \ - "http://127.0.0.1:8080/markdown" - -# All links -curl -s -H "X-Kuri-Session: my-session" \ - "http://127.0.0.1:8080/links" +kuri-skill text +kuri-skill markdown +kuri-skill links ``` ### Accessibility snapshot (interactive refs) ```bash -curl -s -H "X-Kuri-Session: my-session" \ - "http://127.0.0.1:8080/snapshot?filter=interactive&format=compact" +kuri-skill snap ``` -Returns a compact text-tree of interactive elements with refs like `@e0`, `@e1`. +Returns a compact text-tree of interactive elements with refs like `e0`, `e1`. These refs are used as targets for advanced actions (click, type, fill, etc.). **Re-snap after any navigation or DOM change** โ€” refs are snapshot-local and @@ -124,59 +120,42 @@ become stale once the page updates. ## 3. Advanced Operations -Kuri supports many more operations: click, right-click, double-click, hover, -type, fill, select dropdowns, scroll, JavaScript evaluation, cookies, -security audits, HAR recording, tab and session management, and the -experimental `kuri-browser` CLI. - -For full details, read: +For click, type, fill, select, scroll, JavaScript evaluation, cookies, security +audits, HAR recording, and session management, read: ``` skills/pi-kuri-advanced.md ``` -**What's in the advanced file at a glance:** +**What's available at a glance:** | Category | Capabilities | |----------|-------------| | ๐Ÿ–ฑ๏ธ Mouse | click, right-click, double-click, hover | -| โŒจ๏ธ Keyboard | type text, fill input, select option, press keys | -| ๐Ÿ“œ Scroll | up, down, left, right, pixel-amount, element-scroll | +| โŒจ๏ธ Keyboard | type text, fill input, select option | +| ๐Ÿ“œ Scroll | up, down, left, right, pixel-amount | | ๐Ÿง  JS Eval | arbitrary JavaScript, return values | | ๐Ÿช Cookies | list, inspect security flags | | ๐Ÿ”’ Security | audit headers, cookies, HTTPS, mixed content | | ๐Ÿ“ก HAR | HTTP archive recording (start, stop, export) | -| ๐Ÿ“‘ Tabs | create, close, list, switch, multi-tab workflows | -| ๐Ÿ”„ Sessions | X-Kuri-Session grouping, multi-session orchestration | +| ๐Ÿ“‘ Tabs & Sessions | multi-tab, multi-session orchestration | | ๐Ÿงช Experimental | kuri-browser render, bench, parity, serve-cdp | -Read the advanced file only when you need one of those capabilities. The -detailed instructions, endpoint signatures, and workflow examples live there. +Read the advanced file only when you need one of those capabilities. --- ## Tips -- **Prefer `X-Kuri-Session`** over repeating `tab_id` โ€” sessions group tabs - and Kuri tracks the current tab for you. -- **Call `/page/info` before acting** โ€” confirm the right page is loaded. -- **Use `filter=interactive&format=compact`** for snapshot in agent loops. - Reserve `format=json` for programmatic parsing. -- **Read snapshot `state`** before acting on controls โ€” check for - `checked`, `disabled`, `readonly`, `expanded`, `selected` properties. +- **Prefer `kuri-skill` with sessions** โ€” set `KURI_SESSION` env var to group + tabs without repeating `tab_id` on every call. +- **Call `kuri-skill page-info` before acting** โ€” confirm the right page is loaded. - **Treat refs as snapshot-local** โ€” refresh after navigation or DOM updates. -- **`KURI_ALLOW_PRIVATE=1`** โ€” if you need to navigate to localhost or - private IPs during development, set this env var before starting the server. - -## Browser path reference - -- **`kuri` (HTTP API)** โ€” production path for agent loops (this skill). -- **`kuri-fetch`** โ€” standalone no-Chrome text extraction from URLs. -- **`kuri-browse`** โ€” interactive terminal browser. -- **`kuri-agent`** โ€” scriptable CLI automation against the Kuri server. -- **`kuri-browser/`** โ€” experimental Zig-native browser runtime (separate build). +- **`KURI_ALLOW_PRIVATE=1`** โ€” if you need to reach localhost or private IPs + during development, set this before starting the Kuri server. -## Custom project skills +## Plugin reference -Put site-specific workflows (login steps, selectors, data extraction patterns) -in `skills/custom/`. See `skills/custom/hackernews-page-2.md` for an example. +- `skills/pi-kuri-plugin/` โ€” the npm-installable plugin with `kuri-skill` CLI +- `skills/pi-kuri-plugin/package.json` โ€” `pi.skills` field for pi auto-discovery +- `skills/pi-kuri-plugin/SKILL.md` โ€” pi agent skill (synchronized with this file) From e0f0945ba3759ef25249eebaac40700f2a39caa6 Mon Sep 17 00:00:00 2001 From: Robert Maefs Date: Mon, 1 Jun 2026 13:58:38 -0400 Subject: [PATCH 3/8] chore: ignore generated package-lock.json in pi-kuri-plugin --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 13ff498..e4c58ff 100644 --- a/.gitignore +++ b/.gitignore @@ -14,3 +14,4 @@ benchmarks/results/* !benchmarks/results/20260327-111817-https_www.google.com_travel_flights_q=Flights%20to%20TPE%20from%/ !benchmarks/results/20260327-111817-https_www.google.com_travel_flights_q=Flights%20to%20TPE%20from%/** kuri-browser/PARITY.md +skills/pi-kuri-plugin/package-lock.json From 9b6e2d43df1696cae69c07d30bb9403dde98f5da Mon Sep 17 00:00:00 2001 From: Sandcastle Smoke Test Date: Mon, 1 Jun 2026 14:15:59 -0400 Subject: [PATCH 4/8] feat(skill): pi-kuri browser automation skill v2026-06-01 Major improvements to the pi-kuri Pi extension: - Retry wrapper: CDP-fragile endpoints (page/info, screenshot, evaluate, text, markdown, links) auto-retry up to 3 times with exponential backoff on transient 502 failures. - settle_ms param on kuri_screenshot: configurable settle delay before capture for heavy JS pages. - Auto tab_id resolution: kuri_markdown and kuri_links now auto-discover the current session tab when none is provided. - kuri_text fallback: if raw text extraction returns empty, falls back to aria-label traversal of interactive elements. - kuri_console_errors: new diagnostic tool that intercepts console.error/warn and returns the last 30 entries for diagnosing broken pages. --- skills/pi-kuri-plugin/pi-kuri.ts | 1300 ++++++++++++++++++++++++++++++ 1 file changed, 1300 insertions(+) create mode 100644 skills/pi-kuri-plugin/pi-kuri.ts diff --git a/skills/pi-kuri-plugin/pi-kuri.ts b/skills/pi-kuri-plugin/pi-kuri.ts new file mode 100644 index 0000000..4ec5f89 --- /dev/null +++ b/skills/pi-kuri-plugin/pi-kuri.ts @@ -0,0 +1,1300 @@ +/// +import { defineTool, type ExtensionAPI, type ExtensionCommandContext } from "@mariozechner/pi-coding-agent"; +import { Type } from "@sinclair/typebox"; +import { readFileSync } from "node:fs"; +import path from "node:path"; + +/** + * pi-kuri โ€” Kuri browser automation extension for Pi. + * + * Provides tools and commands for driving Kuri's HTTP API: + * - Browser navigation, snapshots, click, type, fill, select, scroll + * - Screenshots, page text, markdown, links extraction + * - Cookie/security audits + * - Session-based agent loops via X-Kuri-Session + * + * Configuration (set via env or pi config): + * KURI_BASE_URL default: http://127.0.0.1:8080 + * KURI_SESSION default: pi-kuri-session + */ + +const PI_KURI_EXTENSION_VERSION = "2026-05-20"; + +// โ”€โ”€ Help text โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +const COMMAND_HELP = [ + "/kuri - open the Kuri command picker (start, navigate, snap, click, audit)", + "/kuri-help - show registered Kuri commands", + `/extension version: ${PI_KURI_EXTENSION_VERSION}`, +]; + +const MENU_CHOICES = [ + "kuri-start - start the Kuri server (if not already running)", + "kuri-health - check if Kuri server is running", + "kuri-tab-new - open a new tab and navigate to a URL", + "kuri-navigate - navigate current tab to a URL", + "kuri-snap - get accessibility snapshot (interactive refs)", + "kuri-page-info - get current page URL, title, ready state", + "kuri-click - click an element by its eN ref", + "kuri-type - type text into an element", + "kuri-fill - fill an input value", + "kuri-select - select a dropdown option", + "kuri-screenshot - capture a screenshot", + "kuri-markdown - get page as markdown", + "kuri-links - extract all links from the current page", + "kuri-text - extract page text", + "kuri-evaluate - execute JavaScript in the page", + "kuri-cookies - list cookies with security flags", + "kuri-headers - check security response headers", + "kuri-audit - run full security audit", + "kuri-back - browser back", + "kuri-forward - browser forward", + "kuri-reload - reload the page", + "kuri-session - set the active Kuri session", + "kuri-base - set the Kuri server base URL", + "kuri-token [token] - show or set the Kuri API auth token", +]; + +// โ”€โ”€ Configuration helpers โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +function kuriBaseUrl(): string { + return process.env.KURI_BASE_URL || "http://127.0.0.1:8080"; +} + +function kuriSession(): string { + return process.env.KURI_SESSION || "pi-kuri-session"; +} + +function kuriAuthToken(): string | undefined { + // 1. Try explicit env vars + if (process.env.KURI_SECRET) return process.env.KURI_SECRET; + if (process.env.KURI_API_TOKEN) return process.env.KURI_API_TOKEN; + // 2. Read from api.token file (auto-generated by new Kuri builds) + try { + const homedir = process.env.HOME || process.env.HOMEPATH || ""; + return readFileSync(`${homedir}/.kuri/api.token`, "utf-8").trim(); + } catch { + return undefined; + } +} + +function kuriHeaders(): Record { + const headers: Record = { + "X-Kuri-Session": kuriSession(), + "Accept": "application/json", + }; + const token = kuriAuthToken(); + if (token) { + headers["Authorization"] = `Bearer ${token}`; + } + return headers; +} + +function parseSiteScalar(value: string): string { + const clean = value.trim().replace(/\s+#.*$/, "").trim(); + if ((clean.startsWith('"') && clean.endsWith('"')) || (clean.startsWith("'") && clean.endsWith("'"))) { + return clean.slice(1, -1); + } + return clean; +} + +function readSiteManifest(projectRoot: string): any { + const manifestPath = path.join(projectRoot, ".sandcastle", "site-manifest.yaml"); + const raw = readFileSync(manifestPath, "utf-8"); + const manifest: any = { pages: {}, auth: {}, kuri: {} }; + let section = ""; + let pageName = ""; + for (const line of raw.split(/\r?\n/)) { + if (!line.trim() || line.trim().startsWith("#")) continue; + const top = line.match(/^([A-Za-z0-9_-]+):\s*(.*)$/); + if (top) { + section = top[1]; + pageName = ""; + if (top[2]) manifest[section] = parseSiteScalar(top[2]); + else if (section === "pages") manifest.pages = manifest.pages || {}; + else if (section === "auth") manifest.auth = manifest.auth || {}; + else if (section === "kuri") manifest.kuri = manifest.kuri || {}; + continue; + } + const nested = line.match(/^\s{2}([A-Za-z0-9_-]+):\s*(.*)$/); + if (nested && section === "pages") { + pageName = nested[1]; + manifest.pages[pageName] = nested[2] ? { url: parseSiteScalar(nested[2]) } : {}; + continue; + } + if (nested && (section === "auth" || section === "kuri")) { + manifest[section][nested[1]] = parseSiteScalar(nested[2]); + continue; + } + const pageField = line.match(/^\s{4}([A-Za-z0-9_-]+):\s*(.*)$/); + if (pageField && section === "pages" && pageName) { + manifest.pages[pageName][pageField[1]] = parseSiteScalar(pageField[2]); + } + } + return manifest; +} + +async function kuriFetch(path: string, query: Record = {}): Promise { + const base = kuriBaseUrl().replace(/\/+$/, ""); + const params = new URLSearchParams(query).toString(); + const url = `${base}${path}${params ? `?${params}` : ""}`; + return fetch(url, { headers: kuriHeaders(), signal: AbortSignal.timeout(30_000) }); +} + +async function kuriFetchText(path: string, query: Record = {}): Promise { + const resp = await kuriFetch(path, query); + if (!resp.ok) { + const body = await resp.text().catch(() => ""); + throw new Error(`Kuri ${path}: ${resp.status} ${resp.statusText} โ€” ${body.slice(0, 200)}`); + } + return resp.text(); +} + +/** + * Retry wrapper for CDP-fragile endpoints (page/info, screenshot, evaluate, markdown, etc.). + * Retries up to maxRetries times with 1s exponential backoff on 502 (CDP failure). + */ +async function kuriFetchTextWithRetry(path: string, query: Record = {}, maxRetries = 2): Promise { + let lastError: Error | null = null; + for (let attempt = 0; attempt <= maxRetries; attempt++) { + try { + return await kuriFetchText(path, query); + } catch (err: any) { + const isCdpFailure = err.message?.includes("502") || err.message?.includes("CDP"); + if (isCdpFailure && attempt < maxRetries) { + lastError = err; + const delay = (attempt + 1) * 1000; + await new Promise(r => setTimeout(r, delay)); + continue; + } + throw err; + } + } + throw lastError || new Error(`Kuri ${path}: max retries exceeded`); +} + +/** + * Resolve the current session tab ID by calling /page/info. + * Returns the tab_id string or throws if unavailable. + */ +async function resolveSessionTabId(): Promise { + const infoText = await kuriFetchTextWithRetry("/page/info", {}); + const info = JSON.parse(infoText); + if (info.tab_id) return String(info.tab_id); + throw new Error("Could not resolve current session tab ID"); +} + +function singleQuotedJsString(value: string): string { + return `'${String(value).replace(/\\/g, "\\\\").replace(/'/g, "\\'").replace(/\r/g, "\\r").replace(/\n/g, "\\n")}'`; +} + +function loginPageWarning(pageInfo: any, text: string, expectedUrl: string): string { + const finalUrl = String(pageInfo?.url || ""); + const title = String(pageInfo?.title || ""); + const lowerText = String(text || "").toLowerCase(); + const compactText = lowerText.replace(/\s+/g, " ").trim(); + const hasLoginAction = /\b(log\s*in|login|sign\s*in|authenticate)\b/.test(`${title} ${compactText}`); + const urlLooksLogin = /\/login\b|login\.php|auth|authenticate/i.test(finalUrl); + const titleLooksLogin = /\b(authenticate|login|sign in)\b/i.test(title); + let expectedHash = ""; + let finalHash = ""; + try { expectedHash = new URL(expectedUrl).hash || ""; } catch { /* ignore */ } + try { finalHash = new URL(finalUrl).hash || ""; } catch { /* ignore */ } + const hashMismatch = Boolean(expectedHash && finalHash && expectedHash !== finalHash); + const littleContent = compactText.length > 0 && compactText.length < 500; + if ((urlLooksLogin || titleLooksLogin || hasLoginAction) && (littleContent || hashMismatch || urlLooksLogin || titleLooksLogin)) { + return "There is strong evidence this capture may be an unauthenticated login page: little page content and login/authentication signals were detected. The reviewer should verify the provided screenshot is for the correct page."; + } + return ""; +} + +async function checkKuriHealth(): Promise { + try { + const resp = await kuriFetch("/health"); + return resp.ok; + } catch { + return false; + } +} + +// โ”€โ”€ Tool definition helper โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +/** + * Wraps defineTool to give `execute` a typed `params` argument. + * The type is inferred from the TypeBox schema via the `S` generic. + */ +function kuriTool>( + def: Omit[0], "parameters" | "execute"> & { + parameters: S; + execute: ( + toolCallId: string, + params: Record, + signal: AbortSignal, + onUpdate: (update: unknown) => void, + ) => Promise | unknown; + }, +) { + return defineTool(def); +} + +// โ”€โ”€ Tool definitions โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +const kuriNavigateTool = kuriTool({ + name: "kuri_navigate", + label: "Navigate Browser", + description: "Navigate a Kuri browser tab to a URL and return the page info.", + promptSnippet: "Navigate browser to a URL", + promptGuidelines: [ + "Use kuri_navigate to open a web page in the Kuri-controlled browser.", + "After navigation, take a snapshot to get interactive element refs.", + ], + parameters: Type.Object({ + url: Type.String({ description: "The URL to navigate to" }), + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID. Defaults to session tab." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = { url: String(params.url) }; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText("/navigate", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriSnapTool = kuriTool({ + name: "kuri_snap", + label: "Browser Snapshot", + description: "Get an accessibility snapshot of the current page with interactive element refs (eN). Use filter=interactive&format=compact for low-token agent loops.", + promptSnippet: "Get browser accessibility snapshot", + promptGuidelines: [ + "Use kuri_snap after navigation to get clickable/typeable element references.", + "Each element has a ref like e0, e1, e2 that can be used with kuri_click, kuri_type, etc.", + "Filter to 'interactive' to get only clickable/typeable elements.", + "Use format 'compact' for minimal token usage.", + "Re-snap after any significant interaction โ€” refs are snapshot-local.", + ], + parameters: Type.Object({ + filter: Type.Optional(Type.String({ description: "Filter: 'interactive' (default) or 'all'" })), + format: Type.Optional(Type.String({ description: "Format: 'compact' (default) or 'json'" })), + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = { + filter: String(params.filter || "interactive"), + format: String(params.format || "compact"), + }; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText("/snapshot", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriClickTool = kuriTool({ + name: "kuri_click", + label: "Click Browser Element", + description: "Click an element in the browser by its accessibility snapshot ref (e.g. e0, e1).", + promptSnippet: "Click browser element by ref", + promptGuidelines: [ + "Use the ref from a kuri_snap snapshot (e.g. e0, e1, e2).", + "Re-snap after clicking to get updated state.", + ], + parameters: Type.Object({ + ref: Type.String({ description: "Element ref from snapshot (e.g. 'e0', 'e1')" }), + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = { action: "click", ref: String(params.ref) }; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText("/action", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriTypeTool = kuriTool({ + name: "kuri_type", + label: "Type in Browser", + description: "Type text into an element by its accessibility snapshot ref.", + promptSnippet: "Type text into browser element", + promptGuidelines: [ + "Use the ref from a kuri_snap snapshot.", + "Clears existing text before typing by default.", + ], + parameters: Type.Object({ + ref: Type.String({ description: "Element ref from snapshot (e.g. 'e0')" }), + value: Type.String({ description: "Text to type into the element" }), + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = { + action: "type", + ref: String(params.ref), + value: String(params.value), + }; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText("/action", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriFillTool = kuriTool({ + name: "kuri_fill", + label: "Fill Input in Browser", + description: "Fill an input element's value (sets value directly, no key events).", + promptSnippet: "Fill browser input element", + parameters: Type.Object({ + ref: Type.String({ description: "Element ref from snapshot (e.g. 'e0')" }), + value: Type.String({ description: "Value to set" }), + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = { + action: "fill", + ref: String(params.ref), + value: String(params.value), + }; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText("/action", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriSelectTool = kuriTool({ + name: "kuri_select", + label: "Select Dropdown Option", + description: "Select an option from a dropdown/select element by ref.", + promptSnippet: "Select dropdown option", + parameters: Type.Object({ + ref: Type.String({ description: "Element ref from snapshot (e.g. 'e0')" }), + value: Type.String({ description: "Option value to select" }), + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = { + action: "select", + ref: String(params.ref), + value: String(params.value), + }; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText("/action", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriPageInfoTool = kuriTool({ + name: "kuri_page_info", + label: "Page Info", + description: "Get current page URL, title, ready state, viewport, and scroll position.", + promptSnippet: "Get current browser page info", + parameters: Type.Object({ + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = {}; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchTextWithRetry("/page/info", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriScreenshotTool = kuriTool({ + name: "kuri_screenshot", + label: "Browser Screenshot", + description: "Capture a browser screenshot (base64-encoded).", + promptSnippet: "Capture browser screenshot", + promptGuidelines: [ + "Use settle_ms when capturing heavy JS-rendered pages to allow content to finish loading.", + ], + parameters: Type.Object({ + format: Type.Optional(Type.String({ description: "Image format: 'png' (default) or 'jpeg'" })), + quality: Type.Optional(Type.Integer({ description: "JPEG quality 1-100 (default: 80, only applies to jpeg)" })), + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + settle_ms: Type.Optional(Type.Number({ description: "Milliseconds to wait after navigation before capture (default: 0). Use on heavy JS pages." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = {}; + if (params.format) query.format = String(params.format); + if (params.quality !== undefined && params.quality !== null) query.quality = String(params.quality); + if (params.tab_id) query.tab_id = String(params.tab_id); + const settleMs = params.settle_ms ? Number(params.settle_ms) : 0; + if (settleMs > 0) { + await new Promise(r => setTimeout(r, settleMs)); + } + const text = await kuriFetchTextWithRetry("/screenshot", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriEvaluateTool = kuriTool({ + name: "kuri_evaluate", + label: "Execute JavaScript", + description: "Execute JavaScript in the browser page and get the result.", + promptSnippet: "Execute JavaScript in browser", + parameters: Type.Object({ + expression: Type.String({ description: "JavaScript expression to evaluate" }), + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = { expression: String(params.expression) }; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchTextWithRetry("/evaluate", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriMarkdownTool = kuriTool({ + name: "kuri_markdown", + label: "Page as Markdown", + description: "Convert the current page to Markdown.", + promptSnippet: "Convert page to markdown", + parameters: Type.Object({ + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID. Auto-resolves if omitted." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = {}; + if (params.tab_id) { + query.tab_id = String(params.tab_id); + } else { + // Auto-resolve current tab since Kuri backend requires tab_id for /markdown + query.tab_id = await resolveSessionTabId(); + } + const text = await kuriFetchTextWithRetry("/markdown", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriLinksTool = kuriTool({ + name: "kuri_links", + label: "Extract Page Links", + description: "Extract all links from the current page.", + promptSnippet: "Extract page links", + parameters: Type.Object({ + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID. Auto-resolves if omitted." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = {}; + if (params.tab_id) { + query.tab_id = String(params.tab_id); + } else { + // Auto-resolve current tab since Kuri backend requires tab_id for /links + query.tab_id = await resolveSessionTabId(); + } + const text = await kuriFetchTextWithRetry("/links", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriTextTool = kuriTool({ + name: "kuri_text", + label: "Extract Page Text", + description: "Extract plain text from the current page. Falls back to aria-labels and visible text if raw text is empty.", + promptSnippet: "Extract page text", + promptGuidelines: [ + "On JS-heavy SPAs, kuri_text may return empty string. The tool auto-falls back to aria-label extraction.", + "If the result still seems sparse, try kuri_snap to see visible interactive elements.", + ], + parameters: Type.Object({ + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = {}; + if (params.tab_id) query.tab_id = String(params.tab_id); + let rawText = await kuriFetchTextWithRetry("/text", query); + let text = rawText; + // Parse the JSON response to check if the extracted value is actually empty + try { + const parsed = JSON.parse(rawText); + const extractedValue = parsed?.result?.result?.value || parsed?.result?.value || ""; + if (extractedValue && String(extractedValue).trim()) { + text = String(extractedValue); + } else { + // Value is empty โ€” trigger fallback + text = ""; + } + } catch { + // Not JSON, use raw + } + // If extracted text is empty or whitespace-only, fall back to aria-labels + // and visible leaf text via evaluate + if (!text || !text.trim()) { + try { + const evalQuery: Record = { + expression: [ + "(function(){", + " var t=[];", + " var els=document.querySelectorAll('[aria-label],[role],a,button');", + " for(var i=0;i { + const query: Record = {}; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText("/cookies", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriAuditTool = kuriTool({ + name: "kuri_audit", + label: "Security Audit", + description: "Run a full security audit on the current page: HTTPS, security headers, cookies exposed to JS.", + promptSnippet: "Run browser security audit", + parameters: Type.Object({ + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = {}; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText("/audit", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriNewTabTool = kuriTool({ + name: "kuri_new_tab", + label: "New Browser Tab", + description: "Create a new browser tab, optionally navigating to a URL.", + promptSnippet: "Create new browser tab", + parameters: Type.Object({ + url: Type.Optional(Type.String({ description: "URL to navigate to in the new tab" })), + activate: Type.Optional(Type.String({ description: "Set as the active/session tab (default: true)" })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = {}; + if (params.url) query.url = String(params.url); + if (params.activate === "false") query.activate = "false"; + const text = await kuriFetchText("/tab/new", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriScrollTool = kuriTool({ + name: "kuri_scroll", + label: "Scroll Browser Page", + description: "Scroll the page or a specific element. Direction: up, down, left, right.", + promptSnippet: "Scroll browser page", + parameters: Type.Object({ + ref: Type.Optional(Type.String({ description: "Element ref to scroll into view (optional). Scrolls page if omitted." })), + direction: Type.Optional(Type.String({ description: "Scroll direction: 'up', 'down', 'left', 'right'" })), + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = {}; + if (params.ref) { query.action = "scrollintoview"; query.ref = String(params.ref); } + if (params.direction) query.direction = String(params.direction); + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText(params.ref ? "/scrollintoview" : "/action", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriScrollUpTool = kuriTool({ + name: "kuri_scroll_up", + label: "Scroll Page Up", + description: "Scroll the browser page up.", + promptSnippet: "Scroll page up", + parameters: Type.Object({ + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = { action: "scroll", direction: "up" }; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText("/action", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriScrollDownTool = kuriTool({ + name: "kuri_scroll_down", + label: "Scroll Page Down", + description: "Scroll the browser page down.", + promptSnippet: "Scroll page down", + parameters: Type.Object({ + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const query: Record = { action: "scroll", direction: "down" }; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchText("/action", query); + return { content: [{ type: "text" as const, text }] }; + }, +}); + +const kuriScreenshotSiteTool = kuriTool({ + name: "kuri_screenshot_site", + label: "Screenshot Authenticated Page (Site Manifest)", + description: "One-shot full flow: use a page from .sandcastle/site-manifest.yaml or an explicit simulator URL, apply repository-defined auth, navigate, settle, and save a screenshot into Sandcastle evidence.", + promptSnippet: "One-shot authenticated screenshot for verifier/reviewer evidence lanes. Prefer page=\"\" or an explicit simulator URL before manual browser tools.", + promptGuidelines: [ + "When your role is responsible for visual evidence and this tool is available, prefer kuri_screenshot_site before trying individual kuri_* tools or gui_* tools.", + "This is the one-shot tool: it handles auth, navigation, settle timing, and PNG saving in a single call.", + "", + "Preferred: use a page name from .sandcastle/site-manifest.yaml:", + ' kuri_screenshot_site page="alert-log"', + ' kuri_screenshot_site page="dashboard"', + "", + "Also valid: use an explicit simulator URL when the ticket needs a per-worktree server:", + ' kuri_screenshot_site url="$(sandstate sim-url KEY)/indexWEB-ODAS.php#alert-log"', + "", + "Avoid output/token unless the operator explicitly asks for an override. Never write evidence to Desktop.", + "By default, the tool saves into .sandcastle/tickets//evidence/ when running inside a task worktree.", + "The tool reports URL/title and warns when the capture looks like an unauthenticated login page; reviewers must inspect that warning before accepting evidence.", + "", + 'Full-page capture: add full_page="true" to capture the entire scrollable document height.', + ' kuri_screenshot_site page="dashboard" full_page="true"', + ' kuri_screenshot_site url="http://..." full_page="true"', + "", + "Do NOT fall back to multi-step kuri_navigate + kuri_snap + kuri_screenshot unless", + "kuri_screenshot_site fails with an error you cannot fix (e.g. auth failure, wrong page).", + "Use gui_screenshot or gui_read only as a fallback when Kuri cannot reach the needed page or state.", + ], + parameters: Type.Object({ + page: Type.Optional(Type.String({ description: "Page name from site manifest (e.g. 'alert-log', 'dashboard', 'networking')" })), + url: Type.Optional(Type.String({ description: "Explicit URL for simulator/per-worktree capture. Prefer page when .sandcastle/site-manifest.yaml has a matching page." })), + project: Type.Optional(Type.String({ description: "Project root path (defaults to CWD). Used to find .sandcastle/site-manifest.yaml" })), + output: Type.Optional(Type.String({ description: "Advanced override. Prefer the default ticket evidence path; never use Desktop." })), + token: Type.Optional(Type.String({ description: "Advanced auth override. Prefer manifest auth and environment configuration." })), + settle_ms: Type.Optional(Type.Number({ description: "Milliseconds to wait after navigation before screenshot (default: from manifest or 3000)" })), + full_page: Type.Optional(Type.Boolean({ + description: "Capture the full scrollable page height (viewport resize + JPEG, default: true). Set to false to capture only the visible viewport.", +})), + tab_id: Type.Optional(Type.String({ description: "Reuse an existing tab ID instead of creating a new one" })), + }, { additionalProperties: false }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const projectRoot = params.project ? String(params.project) : process.cwd(); + const outputPath = params.output ? String(params.output) : ""; + if (outputPath && /(^|\/)Desktop(\/|$)/i.test(outputPath.replace(/\\/g, "/"))) { + return { + content: [{ + type: "text" as const, + text: "Refusing Desktop output path. Visual evidence must stay under the repository Sandcastle evidence area unless the operator explicitly asks for a different non-Desktop path.", + }], + }; + } + const authToken = params.token ? String(params.token) : undefined; + const settleMs = params.settle_ms ? Number(params.settle_ms) : undefined; + const reuseTabId = params.tab_id ? String(params.tab_id) : undefined; + const pageName = params.page ? String(params.page) : ""; + const explicitUrl = params.url ? String(params.url) : ""; + const fullPage = params.full_page !== false; + + // 1. Load site manifest or explicit URL target + let manifest: any = null; + let targetUrl = ""; + let manifestBaseUrl = "http://localhost:8082"; + let manifestSettleMs = 3000; + if (explicitUrl) { + targetUrl = explicitUrl; + try { + manifest = readSiteManifest(projectRoot); + } catch { /* explicit URL can run without a site manifest */ } + try { + const u = new URL(explicitUrl); + manifestBaseUrl = u.origin; + } catch { /* keep default */ } + } else if (pageName) { + try { + manifest = readSiteManifest(projectRoot); + manifestBaseUrl = manifest.base_url || "http://localhost:8082"; + manifestSettleMs = settleMs ?? Number(manifest.kuri?.settle_ms || 3000); + // Normalize: agent might pass underscores instead of hyphens + const pageDef = manifest.pages?.[pageName] || manifest.pages?.[pageName.replace(/_/g, "-")]; + if (!pageDef) { + const available = Object.keys(manifest.pages || {}).join(", "); + return { + content: [{ type: "text" as const, text: `Unknown page '${pageName}'. Available: ${available}` }], + }; + } + targetUrl = manifestBaseUrl.replace(/\/$/, "") + pageDef.url; + } catch (err: any) { + return { + content: [{ type: "text" as const, text: `Failed to load site manifest: ${err.message}` }], + }; + } + } else { + return { + content: [{ type: "text" as const, text: 'Must provide page="" from .sandcastle/site-manifest.yaml or url="".' }], + }; + } + + // 2. Read auth config + const authMethod = manifest?.auth?.method || "none"; + const tokenEnv = manifest?.auth?.token_env || "DEV_SCREENSHOT_TOKEN"; + const defaultToken = manifest?.auth?.default_token || ""; + const loginPath = manifest?.auth?.login_url || manifest?.auth?.login_page || ""; + const username = manifest?.auth?.username || ""; + const password = manifest?.auth?.password || ""; + const usernameSelector = manifest?.auth?.username_selector || "input[name=\"authName\"]"; + const passwordSelector = manifest?.auth?.password_selector || "input[name=\"authPassword\"]"; + const submitSelector = manifest?.auth?.submit_selector || "button[type=\"submit\"]"; + const resolvedToken = authToken || process.env[tokenEnv] || defaultToken; + + // 3. Ensure Kuri is healthy + const healthResp = await kuriFetch("/health"); + if (!healthResp.ok) { + return { + content: [{ type: "text" as const, text: `Kuri server not responding (${healthResp.status}) โ€” start it first with 'kuri-start' or 'KURI_ALLOW_PRIVATE=1 kuri'` }], + }; + } + + // 4. Get or create tab + let tabId = reuseTabId || ""; + let newTabResp = ""; + if (!tabId) { + newTabResp = await kuriFetchText("/tab/new", { url: "about:blank", activate: "true" }); + try { + const tabData = JSON.parse(newTabResp); + tabId = tabData.tab_id || tabData.id || ""; + } catch { /* fall through */ } + } + if (!tabId) { + return { + content: [{ type: "text" as const, text: `Failed to create Kuri tab. Kuri may not be running. Response: ${newTabResp.slice(0, 200)}` }], + }; + } + + // 5. Auth is repository-owned manifest behavior, not pi-dev product knowledge. + if (authMethod === "form-login") { + const originUrl = manifestBaseUrl.replace(/\/$/, ""); + const loginPageUrl = loginPath.startsWith("http") ? loginPath : originUrl + (loginPath || "/login.php"); + + try { + await kuriFetchText("/navigate", { tab_id: tabId, url: loginPageUrl }); + await new Promise(r => setTimeout(r, 1500)); + + await kuriFetchText("/evaluate", { + tab_id: tabId, + expression: `document.querySelector(${singleQuotedJsString(usernameSelector)}).value=${JSON.stringify(username)}`, + }); + await kuriFetchText("/evaluate", { + tab_id: tabId, + expression: `document.querySelector(${singleQuotedJsString(passwordSelector)}).value=${JSON.stringify(password)}`, + }); + await new Promise(r => setTimeout(r, 300)); + + await kuriFetchText("/evaluate", { + tab_id: tabId, + expression: `document.querySelector(${singleQuotedJsString(submitSelector)}).click()`, + }); + + await new Promise(r => setTimeout(r, 3000)); + } catch (err: any) { + return { + content: [{ type: "text" as const, text: `Login failed (${err.message}). The captured screenshot may show an unauthenticated page.` }], + }; + } + } else if (authMethod === "bearer-token-session" && resolvedToken && loginPath) { + const originUrl = manifestBaseUrl.replace(/\/$/, ""); + const loginPageUrl = loginPath.startsWith("http") ? loginPath : originUrl + loginPath; + await kuriFetchText("/navigate", { + tab_id: tabId, + url: loginPageUrl, + headers: JSON.stringify({ Authorization: `Bearer ${resolvedToken}` }), + }); + await new Promise(r => setTimeout(r, 1500)); + } + + // 6. Navigate to target + await kuriFetchText("/navigate", { + tab_id: tabId, + url: targetUrl, + }); + + // 7. Settle + const settle = settleMs ?? manifestSettleMs; + await new Promise(r => setTimeout(r, settle)); + + // 7.5 Full-page mode: resize viewport to full page height so the screenshot + // captures the entire scrollable document instead of just the viewport. + // Uses JPEG format for tall pages since Chrome's PNG encoder can OOM. + // Resets viewport after capture. + let originalViewport: { w: number; h: number } | null = null; + let useJpegForTall = false; + if (fullPage) { + try { + const piText = await kuriFetchText("/page/info", { tab_id: tabId }).catch(() => "{}"); + const pi: any = JSON.parse(piText); + const vw = Number(pi.viewport_width) || 1920; + const vh = Number(pi.viewport_height) || 1080; + originalViewport = { w: vw, h: vh }; + const dimText = await kuriFetchText("/evaluate", { + tab_id: tabId, + expression: "JSON.stringify({sw:document.documentElement.scrollWidth,sh:document.documentElement.scrollHeight})", + }).catch(() => ""); + const dimJson: any = JSON.parse(dimText); + const sw = Number(dimJson?.result?.result?.value ? JSON.parse(dimJson.result.result.value).sw : 0) || vw; + const sh = Number(dimJson?.result?.result?.value ? JSON.parse(dimJson.result.result.value).sh : 0) || vh; + if (sh > vh && sh > 0 && sw > 0) { + useJpegForTall = true; + await kuriFetch("/set/viewport", { tab_id: tabId, width: String(sw), height: String(sh) }); + // Let the renderer catch up after resize + await new Promise(r => setTimeout(r, 1000)); + } + } catch { + // Best-effort: if viewport resize fails, fall through to normal screenshot + } + } + + // 8. Capture advisory page state before screenshot so the reviewer can tell + // whether auth landed on the expected page or a login wall. + const pageInfoText = await kuriFetchText("/page/info", { tab_id: tabId }).catch(() => ""); + let pageInfo: any = {}; + try { + pageInfo = pageInfoText ? JSON.parse(pageInfoText) : {}; + } catch { /* advisory only */ } + const pageText = await kuriFetchText("/text", { tab_id: tabId }).catch(() => ""); + const warning = loginPageWarning(pageInfo, pageText, targetUrl); + + // 9. Screenshot โ€” use JPEG for tall pages (full-page mode) since + // Chrome's PNG encoder may OOM on very tall viewport captures + const screenshotResp = await kuriFetch("/screenshot", { + tab_id: tabId, + ...(useJpegForTall ? { format: "jpeg", quality: "30" } : {}), + }); + const screenshotJson = await screenshotResp.json(); + let pngData: string | undefined; + if (screenshotJson.result?.result?.data) { + pngData = screenshotJson.result.result.data; + } else if (screenshotJson.result?.data) { + pngData = screenshotJson.result.data; + } else if (screenshotJson.data) { + pngData = screenshotJson.data; + } + + if (!pngData) { + return { + content: [{ type: "text" as const, text: `Screenshot failed: Kuri returned an unexpected response format. The page may not have loaded correctly. ${JSON.stringify(screenshotJson).slice(0, 200)}` }], + }; + } + + // 10. Save PNG โ€” default to ticket-owned Sandcastle evidence, with explicit + // non-Desktop override allowed. + const pageSlug = pageName || targetUrl.replace(/[^a-z0-9]/gi, "-").replace(/-+/g, "-").slice(0, 40).toLowerCase(); + let savedPath = outputPath; + let repoRoot = ""; + let taskKey = ""; + if (!savedPath) { + const wtMatch = projectRoot.match(/(.+?)\/\.sandcastle\/worktrees\/sandcastle-([A-Z0-9]{2,5})-/i); + if (wtMatch) { + repoRoot = wtMatch[1]; + taskKey = wtMatch[2].toLowerCase(); + const stamp = new Date().toISOString().replace(/[:.]/g, "-"); + savedPath = path.join(repoRoot, ".sandcastle", "tickets", taskKey, "evidence", `kuri-${pageSlug}-${stamp}`, "visual-evidence.png"); + } else { + savedPath = path.join(projectRoot, ".sandcastle", "evidence", "manual", `evidence-${pageSlug}-${Date.now()}.png`); + } + } + const buf = Buffer.from(pngData, "base64"); + const { mkdirSync, writeFileSync } = require("node:fs") as typeof import("node:fs"); + mkdirSync(path.dirname(savedPath), { recursive: true }); + writeFileSync(savedPath, buf); + + // 10.5 Full-page mode: reset viewport to original dimensions + if (originalViewport) { + kuriFetch("/set/viewport", { + tab_id: tabId, + width: String(originalViewport.w), + height: String(originalViewport.h), + }).catch(() => {}); + } + + // Warn if screenshot is suspiciously small (blank/error page) + if (buf.length < 2000) { + return { + content: [{ type: "text" as const, text: `Evidence file ${savedPath} saved (only ${buf.length} bytes โ€” page may be blank or returned an error). Inspect before relying on this evidence.` }], + }; + } + + let pageTitle = ""; + let pageUrl = ""; + pageTitle = pageInfo.title || ""; + pageUrl = pageInfo.url || ""; + const artifactPath = repoRoot && savedPath.startsWith(repoRoot) + ? path.relative(repoRoot, savedPath) + : savedPath; + const evidenceManifest = { + status: "passed", + role: "reviewer", + screen: targetUrl, + finalUrl: pageUrl, + title: pageTitle, + page: pageName, + capturedAt: new Date().toISOString(), + artifacts: [artifactPath], + checks: [ + { name: "visual evidence", passed: true, detail: "Kuri screenshot captured" }, + { name: "page rendered", passed: true, detail: "document body exists" }, + ], + warnings: warning ? [warning] : [], + ...(taskKey ? { taskKey: taskKey.toUpperCase() } : {}), + }; + const manifestJson = `${JSON.stringify(evidenceManifest, null, 2)}\n`; + writeFileSync(path.join(path.dirname(savedPath), "visual-evidence.json"), manifestJson); + writeFileSync(path.join(path.dirname(savedPath), "visual-evidence-manifest.json"), manifestJson); + + return { + content: [{ + type: "text" as const, + text: [ + `Evidence file ${savedPath} saved (${buf.length} bytes)`, + `Manifest ${path.join(path.dirname(savedPath), "visual-evidence-manifest.json")} saved`, + outputPath ? "Note: output override was used; prefer the default Sandcastle evidence path for ticket handoff." : "", + authToken ? "Note: token override was used; prefer repository manifest auth/env config." : "", + warning ? `Warning: ${warning}` : "", + `Page: ${pageTitle}`, + `URL: ${pageUrl}`, + ].filter(Boolean).join("\n"), + }], + }; + }, +}); + +const kuriConsoleErrorsTool = kuriTool({ + name: "kuri_console_errors", + label: "Console Errors", + description: "Extract recent console.error/warn entries from the browser page. Useful for diagnosing why a page isn't rendering.", + promptSnippet: "Get browser console errors", + promptGuidelines: [ + "Use after a page loads but seems empty or broken โ€” checks for JS errors.", + "If the result is empty, no errors were captured (or the console was cleared).", + ], + parameters: Type.Object({ + tab_id: Type.Optional(Type.String({ description: "Optional explicit tab ID." })), + }), + execute: async (_toolCallId, params, _signal, _onUpdate) => { + const expression = [ + "(function(){", + " var e = window.__kuriErrorLog || [];", + " if (!window.__kuriErrorLog) {", + " window.__kuriErrorLog = [];", + " var orig = console.error;", + " console.error = function() {", + " var args = Array.prototype.slice.call(arguments);", + " window.__kuriErrorLog.push(args.map(function(a){return typeof a === 'string' ? a : JSON.stringify(a)}).join(' '));", + " return orig.apply(console, arguments);", + " };", + " var origWarn = console.warn;", + " console.warn = function() {", + " var args = Array.prototype.slice.call(arguments);", + " window.__kuriErrorLog.push('[WARN] ' + args.map(function(a){return typeof a === 'string' ? a : JSON.stringify(a)}).join(' '));", + " return origWarn.apply(console, arguments);", + " };", + " }", + " return e.length ? e.slice(-30).join('\\n') : '';", + " })()", + ].join(""); + const query: Record = { expression }; + if (params.tab_id) query.tab_id = String(params.tab_id); + const text = await kuriFetchTextWithRetry("/evaluate", query); + let result = ""; + try { + const parsed = JSON.parse(text); + result = parsed?.result?.result?.value || parsed?.result?.value || ""; + } catch { + result = text; + } + if (!result || !String(result).trim()) { + result = "No console errors captured (console.error interceptor installed)."; + } + return { content: [{ type: "text" as const, text: String(result) }] }; + }, +}); + +const allTools = [ + kuriNavigateTool, + kuriSnapTool, + kuriClickTool, + kuriTypeTool, + kuriFillTool, + kuriSelectTool, + kuriPageInfoTool, + kuriScreenshotTool, + kuriEvaluateTool, + kuriMarkdownTool, + kuriLinksTool, + kuriTextTool, + kuriConsoleErrorsTool, + kuriCookiesTool, + kuriAuditTool, + kuriNewTabTool, + kuriScrollTool, + kuriScrollUpTool, + kuriScrollDownTool, + kuriScreenshotSiteTool, +]; + +// โ”€โ”€ Slash command handlers โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +function isDuplicateToolConflict(error: unknown): boolean { + return error instanceof Error && !!error.message?.includes("already defined"); +} + +async function cmdKuriStart(_args: string, ctx: ExtensionCommandContext): Promise { + const ok = await checkKuriHealth(); + if (ok) { + ctx.ui.notify("Kuri server is already running."); + return; + } + ctx.ui.notify("Kuri is not running. Start it manually:\n kuri\n\nOr with options:\n PORT=8080 kuri"); +} + +async function cmdKuriHealth(_args: string, ctx: ExtensionCommandContext): Promise { + const ok = await checkKuriHealth(); + if (ok) { + const resp = await kuriFetch("/health"); + const data = await resp.text(); + ctx.ui.notify(`Kuri server is running.\n${data.slice(0, 500)}`); + } else { + ctx.ui.notify("Kuri server is NOT running. Start it with: kuri"); + } +} + +function cmdKuriTabNew(args: string, ctx: ExtensionCommandContext): void { + ctx.ui.notify(`Use the kuri_new_tab tool with url="${args.trim() || ''}"`); +} + +function cmdKuriNavigate(args: string, ctx: ExtensionCommandContext): void { + ctx.ui.notify(`Use the kuri_navigate tool with url="${args.trim() || ''}"`); +} + +function cmdKuriSnap(_args: string, ctx: ExtensionCommandContext): void { + ctx.ui.notify("Use the kuri_snap tool with filter=interactive&format=compact"); +} + +function cmdBase(args: string, ctx: ExtensionCommandContext): void { + const url = args.trim(); + if (url) { + process.env.KURI_BASE_URL = url; + ctx.ui.notify(`KURI_BASE_URL set to ${url}`); + } else { + ctx.ui.notify(`Current KURI_BASE_URL: ${kuriBaseUrl()}`); + } +} + +function cmdSession(args: string, ctx: ExtensionCommandContext): void { + const id = args.trim(); + if (id) { + process.env.KURI_SESSION = id; + ctx.ui.notify(`KURI_SESSION set to ${id}`); + } else { + ctx.ui.notify(`Current KURI_SESSION: ${kuriSession()}`); + } +} + +function cmdToken(args: string, ctx: ExtensionCommandContext): void { + const token = args.trim(); + if (token) { + process.env.KURI_SECRET = token; + ctx.ui.notify("KURI_SECRET (API auth token) set."); + } else { + const current = kuriAuthToken(); + if (current) { + ctx.ui.notify(`Auth token: ${current.slice(0, 20)}...`); + } else { + ctx.ui.notify("No auth token found. Set via KURI_SECRET env var or ~/.kuri/api.token file."); + } + } +} + +async function cmdGenericKuri(action: string, _args: string, ctx: ExtensionCommandContext): Promise { + const ok = await checkKuriHealth(); + if (!ok) { + ctx.ui.notify("Kuri server is not running. Start it first with 'kuri'."); + return; + } + ctx.ui.notify(`Run: kuri_${action} tool`); +} + +// โ”€โ”€ Extension entry point โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +export default function kuriCommands(pi: ExtensionAPI): void { + for (const tool of allTools) { + try { + pi.registerTool(tool); + } catch (error) { + if (!isDuplicateToolConflict(error)) { + throw error; + } + } + } + + // Register /kuri-photo slash command for easy one-shot screenshots + pi.registerCommand("kuri-photo", { + description: "One-shot site-manifest screenshot. Usage: /kuri-photo [project-root]", + handler: async (args, ctx) => { + const parts = args.trim().split(/\s+/); + const pageName = parts[0] || ""; + const projectRoot = parts[1] || process.cwd(); + ctx.ui.notify(`Use the kuri_screenshot_site tool with page="${pageName}" project="${projectRoot}"`); + }, + }); + + pi.on("session_start", async (_event, ctx) => { + ctx.ui.setStatus("kuri", `๐ŸŒฐ Kuri ${PI_KURI_EXTENSION_VERSION}`); + }); + + pi.on("session_shutdown", async (_event, ctx) => { + ctx.ui.setStatus("kuri", undefined); + }); + + pi.on("before_agent_start", async (event) => { + const baseUrl = kuriBaseUrl(); + const sessionId = kuriSession(); + const kuriGuide = [ + "", + `## Kuri Browser Automation (extension ${PI_KURI_EXTENSION_VERSION})`, + "", + "Kuri is a Zig-native browser automation server. Use it only when your role's tool list includes Kuri tools.", + "If your role does not include `kuri_screenshot_site`, do not attempt browser capture; record the target page, selector, or URL for the verifier/reviewer evidence lane.", + "", + "### 1. One-Shot Screenshot (PREFERRED)", + "", + 'Verifier/reviewer lanes should use `kuri_screenshot_site` for visual evidence. This handles auth, navigation,', + "settle timing, and evidence PNG saving in a single call:", + "", + ' kuri_screenshot_site page="alert-log" # from .sandcastle/site-manifest.yaml', + ' kuri_screenshot_site page="dashboard"', + "", + "For per-worktree simulators, an explicit URL is also valid:", + ' kuri_screenshot_site url="$(sandstate sim-url KEY)/indexWEB-ODAS.php#networking"', + "", + "Avoid output/token unless the operator explicitly asks for an override.", + "Never write evidence to Desktop; default output stays under .sandcastle evidence paths.", + "Read the tool response after capture. If it warns that the page looks like an unauthenticated login page, verify or retry before treating the screenshot as proof.", + "", + "When Kuri tools are available, use this first before manual browser tools or gui_* tools.", + "", + "### 2. Manual Browser Tools (fallback only if screenshot_site fails)", + "", + "Only use these when kuri_screenshot_site cannot reach the right page or state.", + "", + "**Open a tab:**", + " kuri_new_tab(url?)", + "", + "**Check what's on the page:**", + " kuri_page_info โ€” URL, title, ready state", + " kuri_snap โ€” interactive element refs (e0, e1...)", + " kuri_screenshot โ€” capture screenshot to inspect", + "", + "**Act on elements (by ref from snap):**", + " kuri_click(ref) โ€” click element", + " kuri_type(ref, value) โ€” type text", + " kuri_fill(ref, value) โ€” fill input value", + " kuri_select(ref, value) โ€” select dropdown option", + " kuri_scroll(direction?) โ€” scroll page", + "", + "**Get page content:**", + " kuri_markdown โ€” page as markdown (auto-resolves tab)", + " kuri_links โ€” all links (auto-resolves tab)", + " kuri_text โ€” plain text (falls back to aria-labels on SPAs)", + " kuri_evaluate(expression) โ€” run JavaScript", + " kuri_console_errors โ€” diagnose JS errors when page is empty/broken", + "", + "**Tips:**", + " - CDP-fragile endpoints auto-retry (up to 3 attempts) on transient failures.", + " - Heavy JS pages: add settle_ms=5000 to kuri_screenshot for settle time.", + " - Re-snap after navigation or DOM changes โ€” refs are snapshot-local.", + "", + ].join("\n"); + return { + systemPrompt: `${event.systemPrompt}\n${kuriGuide}`, + }; + }); + + pi.registerCommand("kuri", { + description: "Open the Kuri command picker", + handler: async (_args, ctx) => { + const choice = await ctx.ui.select("Kuri commands", MENU_CHOICES); + if (!choice) return; + const prefix = choice.split(" - ")[0]; + ctx.ui.setStatus("kuri", `selected: ${prefix}`); + }, + }); + + pi.registerCommand("kuri-help", { + description: "Show registered Kuri commands", + handler: async (_args, ctx) => { + ctx.ui.notify(COMMAND_HELP.join("\n")); + }, + }); + + pi.registerCommand("kuri-start", { + description: "Check or start the Kuri server", + handler: cmdKuriStart, + }); + + pi.registerCommand("kuri-health", { + description: "Check Kuri server health", + handler: cmdKuriHealth, + }); + + pi.registerCommand("kuri-tab-new", { + description: "Open a new browser tab and optionally navigate to a URL", + handler: cmdKuriTabNew, + }); + + pi.registerCommand("kuri-navigate", { + description: "Navigate the current tab to a URL", + handler: cmdKuriNavigate, + }); + + pi.registerCommand("kuri-snap", { + description: "Get accessibility snapshot with interactive refs", + handler: cmdKuriSnap, + }); + + pi.registerCommand("kuri-session", { + description: "Set the active Kuri session ID", + handler: cmdSession, + }); + + pi.registerCommand("kuri-base", { + description: "Set the Kuri server base URL", + handler: cmdBase, + }); + + pi.registerCommand("kuri-token", { + description: "Show or set the Kuri API auth token", + handler: cmdToken, + }); + + // Register convenience forwarders for common commands + for (const action of ["page-info", "click", "type", "fill", "select", "screenshot", + "markdown", "links", "text", "evaluate", "cookies", "audit", + "back", "forward", "reload"]) { + const cmdName = `kuri-${action}`; + const handler = async (_args: string, ctx: ExtensionCommandContext) => { + await cmdGenericKuri(action.replace("-", "_"), _args, ctx); + }; + pi.registerCommand(cmdName, { + description: `Browser: ${action}`, + handler, + }); + } +} From c4b6ac97e89142308c1173d4febef02e3a755ee9 Mon Sep 17 00:00:00 2001 From: Sandcastle Smoke Test Date: Mon, 1 Jun 2026 14:18:08 -0400 Subject: [PATCH 5/8] feat(plugin): add pi-kuri.ts extension with CDP retry, settle_ms, tab auto-resolve, text fallback, console_errors --- skills/pi-kuri-plugin/README.md | 13 +++++++++++++ skills/pi-kuri-plugin/SKILL.md | 28 +++++++++++++++++++++++++--- 2 files changed, 38 insertions(+), 3 deletions(-) diff --git a/skills/pi-kuri-plugin/README.md b/skills/pi-kuri-plugin/README.md index e18fcd3..9ac51de 100644 --- a/skills/pi-kuri-plugin/README.md +++ b/skills/pi-kuri-plugin/README.md @@ -5,6 +5,9 @@ screenshot, extract page content, and interact with web pages via Kuri's HTTP AP This package provides: +- **`pi-kuri.ts`** โ€” Pi agent extension registering Kuri tools (`kuri_navigate`, + `kuri_snap`, `kuri_screenshot`, `kuri_text`, `kuri_evaluate`, `kuri_click`, + `kuri_console_errors`, etc.) for agent-driven browser automation - **`kuri-skill` CLI** โ€” a portable Node.js wrapper around Kuri's HTTP API - **`SKILL.md`** โ€” pi.dev agent skill with two-tier progressive disclosure - **`references/ADVANCED.md`** โ€” on-demand reference for click, type, JS eval, etc. @@ -28,6 +31,16 @@ pi install npm:pi-kuri-skill npm install -g pi-kuri-skill ``` +### Extension setup + +To make the Pi extension available to your agent: + +```bash +ln -s $(pwd)/pi-kuri.ts ~/.pi/agent/extensions/pi-kuri.ts +``` + +Then restart pi.dev. The agent will have all Kuri tools registered. + Then use the `kuri-skill` command: ```bash diff --git a/skills/pi-kuri-plugin/SKILL.md b/skills/pi-kuri-plugin/SKILL.md index 3d110d4..df5e5b1 100644 --- a/skills/pi-kuri-plugin/SKILL.md +++ b/skills/pi-kuri-plugin/SKILL.md @@ -5,9 +5,31 @@ description: Kuri browser automation for pi.dev agents โ€” navigate, screenshot, # Pi Kuri Plugin -Kuri is a Zig-native browser automation server. This plugin provides a CLI -wrapper (`kuri-skill`) and on-demand reference docs for agent-driven browser -automation via Kuri's HTTP API. +Kuri is a Zig-native browser automation server. This plugin provides: +- A Pi agent extension (`pi-kuri.ts`) with registered tools for agent-driven + browser automation โ€” navigate, screenshot, snap, click, type, evaluate, etc. +- A CLI wrapper (`kuri-skill`) for human use in the terminal +- On-demand reference docs for both tiers + +### Two Ways to Use Kuri + +| Method | What it gives you | When to use | +|--------|-------------------|-------------| +| **Pi extension** ๐Ÿง  | Registered tools: `kuri_navigate`, `kuri_snap`, `kuri_screenshot`, `kuri_text`, `kuri_evaluate`, `kuri_click`, `kuri_console_errors`, etc. | The agent drives the browser directly via tool calls | +| **CLI** โŒจ๏ธ | `kuri-skill` command: navigate, screenshot, text, markdown, snap, click, type | Human-in-the-loop or scripts | + +The extension is the primary interface for pi.dev agents. The CLI is for +manual use, debugging, and scripting. + +### Install the Extension + +Symlink `pi-kuri.ts` into your pi.dev agent extensions directory: + +```bash +ln -s $(pwd)/pi-kuri.ts ~/.pi/agent/extensions/pi-kuri.ts +``` + +After restarting pi.dev, the agent will have all Kuri tools available. ### Getting `kuri-skill` on PATH From 215109a30a66a7bf55c72949059258fcf311ee5f Mon Sep 17 00:00:00 2001 From: Sandcastle Smoke Test Date: Mon, 1 Jun 2026 14:24:27 -0400 Subject: [PATCH 6/8] fix(plugin): address Copilot CR items #1, #2, #5, #6, #7, #8 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - #1: Fix action arg parsing โ€” dispatch by subcommand schema instead of naive sequential fill. click, type, fill, select, scroll, evaluate, close-tab each parse positional args correctly. - #2: Fix endpoint mismatch โ€” CLI now calls /page/info (server endpoint) instead of /page-info, removing degraded /tabs fallback. - #5: Use POSIX 'command -v' instead of non-portable 'which'. - #6: Remove unused imports (mkdirSync, createWriteStream, get). - #7: Remove double-encoding via encodeURI in cmdTabNew. - #8: Replace duplicate ADVANCED.md copy with symlink to canonical pi-kuri-advanced.md to prevent drift. --- skills/pi-kuri-plugin/references/ADVANCED.md | 355 +------------------ skills/pi-kuri-plugin/scripts/kuri.js | 48 +-- 2 files changed, 25 insertions(+), 378 deletions(-) mode change 100644 => 120000 skills/pi-kuri-plugin/references/ADVANCED.md diff --git a/skills/pi-kuri-plugin/references/ADVANCED.md b/skills/pi-kuri-plugin/references/ADVANCED.md deleted file mode 100644 index 8d9ff81..0000000 --- a/skills/pi-kuri-plugin/references/ADVANCED.md +++ /dev/null @@ -1,354 +0,0 @@ ---- -name: pi-kuri-advanced -description: Advanced Kuri browser automation for pi.dev agents โ€” click, type, fill, select, scroll, JavaScript evaluation, cookies, security audits, session management, HAR recording, and the experimental kuri-browser CLI. Load this on demand when the core skill (`skills/pi-kuri-skills.md`) is not enough. ---- - -# Pi Kuri Advanced Reference - -This file covers operations beyond the core skill (`skills/pi-kuri-skills.md`). -Load it when you need interactive actions, JavaScript evaluation, cookies, -security audits, session management, or experimental features. - -All commands use the `kuri-skill` CLI from `skills/pi-kuri-plugin/`. If -`kuri-skill` is not on your PATH, run commands via: - -```bash -node skills/pi-kuri-plugin/scripts/kuri.js -``` - -Set `KURI_SESSION` to group tabs without repeating `tab_id` on every call: - -```bash -export KURI_SESSION=my-session -``` - ---- - -## ๐Ÿ–ฑ๏ธ Mouse Actions - -### Click an element - -```bash -kuri-skill action click e1 -kuri-skill action click ref=e1 tab_id=TABID123 - -# Right-click -kuri-skill action click ref=e1 button=right - -# Double-click -kuri-skill action click ref=e1 clickCount=2 -``` - -### Hover - -Use the snapshot to detect hover-triggered UI, then evaluate JS for precise -hover scenarios: - -```bash -kuri-skill action evaluate \ - expression="document.querySelector('button').dispatchEvent(new MouseEvent('mouseover',{bubbles:true}))" -``` - ---- - -## โŒจ๏ธ Keyboard Actions - -### Type text into an input - -```bash -kuri-skill action type e2 "hello world" -kuri-skill action type ref=e2 value="hello world" -``` - -### Fill input value - -```bash -kuri-skill action fill e2 "user@example.com" -kuri-skill action fill ref=e2 value="user@example.com" -``` - -### Select a dropdown option - -```bash -kuri-skill action select e3 "option-value" -kuri-skill action select ref=e3 value="option-value" -``` - ---- - -## ๐Ÿ“œ Scrolling - -```bash -# Scroll down (default) -kuri-skill action scroll - -# Scroll up -kuri-skill action scroll direction=up amount=10 - -# Scroll right -kuri-skill action scroll direction=right amount=3 - -# Scroll by pixel amount -kuri-skill action scroll direction=down amount=500 -``` - ---- - -## ๐Ÿง  JavaScript Evaluation - -```bash -# Get page title -kuri-skill action evaluate expression="document.title" - -# Read heading text -kuri-skill action evaluate \ - expression="document.querySelector('h1').textContent" - -# Extract dynamic state -kuri-skill action evaluate \ - expression="JSON.stringify(window.__INITIAL_STATE__)" - -# Get page metrics -kuri-skill action evaluate \ - expression="document.body.scrollHeight" - -# Check element visibility -kuri-skill action evaluate \ - expression="document.querySelector('.spinner').offsetParent!==null" -``` - -Returns the evaluated result as a string, number, boolean, or JSON value. - ---- - -## ๐Ÿช Cookies - -```bash -kuri-skill action cookies -``` - -Returns an array of cookies with: `name`, `value`, `domain`, `path`, `secure`, -`httpOnly`, `sameSite`, `expires`. - ---- - -## ๐Ÿ”’ Security Audit - -```bash -kuri-skill action audit -``` - -Checks performed: -- Security response headers: HSTS, CSP, X-Frame-Options, X-Content-Type-Options -- Cookie security: Secure flag, HttpOnly flag, SameSite attribute -- HTTPS enforcement -- Mixed content warnings - ---- - -## ๐Ÿ“ก HAR Recording (HTTP Archive) - -Record network activity for API discovery or debugging: - -```bash -# Start recording -kuri-skill action har-start - -# Perform actions that make network requests... - -# Stop and retrieve the HAR log -kuri-skill action har-stop - -# Get current HAR log without stopping -kuri-skill action har -``` - -Useful for reverse-engineering API calls or debugging network issues. - ---- - -## ๐Ÿ“‘ Tab & Session Management - -```bash -# List all tabs -kuri-skill tabs - -# Create a new tab -kuri-skill tab-new https://example.com - -# Close a tab -kuri-skill action close-tab TABID123 - -# Browser back -kuri-skill action back - -# Browser forward -kuri-skill action forward - -# Reload -kuri-skill action reload -``` - -### Multiple sessions - -Run independent browser sessions by setting `KURI_SESSION` to different values: - -```bash -KURI_SESSION=session-a kuri-skill tab-new https://example.com -KURI_SESSION=session-b kuri-skill tab-new https://other.com -``` - -Each session tracks its own tabs and current tab independently. - ---- - -## ๐Ÿงช Experimental โ€” kuri-browser - -The `kuri-browser/` subdirectory is an **experimental** Zig-native browser -runtime with its own rendering engine (separate build, not wired into Kuri's -root build). Use for evaluation and development only. - -```bash -cd kuri-browser -zig build run -- render https://news.ycombinator.com --selector ".titleline a" --dump text -zig build run -- render https://todomvc.com/examples/react/dist/ --js --wait-eval "..." -zig build run -- bench --offline -zig build run -- serve-cdp --port 9333 -``` - -Screenshots fall back to the main Kuri/CDP renderer โ€” start the normal Kuri -server first, then: - -```bash -cd kuri-browser -zig build run -- screenshot https://example.com --out example.jpg \ - --compress --kuri-base http://127.0.0.1:8080 -``` - ---- - -## Common Workflow Patterns - -### Pattern 1: Quick screenshot verification - -```bash -export KURI_SESSION=verify - -kuri-skill tab-new https://example.com -kuri-skill page-info -kuri-skill screenshot # set KURI_OUTPUT to customize path -``` - -### Pattern 2: Fill form and submit - -```bash -export KURI_SESSION=login - -kuri-skill tab-new https://example.com/login -kuri-skill snap # Get refs - -kuri-skill action type e0 "user@example.com" -kuri-skill action type e1 "password123" -kuri-skill action click e2 # Submit button - -kuri-skill page-info # Check result -``` - -### Pattern 3: Extract dynamic content - -```bash -export KURI_SESSION=extract - -kuri-skill tab-new https://example.com/app -kuri-skill action evaluate expression="JSON.stringify(window.__DATA__)" -kuri-skill text # Also get rendered text -``` - -### Pattern 4: Multi-step interaction loop - -```bash -export KURI_SESSION=search - -# 1. Open page -kuri-skill tab-new https://example.com/search - -# 2. Get snapshot for refs -kuri-skill snap - -# 3. Type search query -kuri-skill action type e0 "query text" - -# 4. Check for results via JS -kuri-skill action evaluate \ - expression="document.querySelectorAll('.result').length" - -# 5. Re-snap after DOM change -kuri-skill snap - -# 6. Click a result -kuri-skill action click e3 - -# 7. Verify new page -kuri-skill page-info - -# 8. Screenshot for evidence -kuri-skill screenshot -``` - ---- - -## Quick Parameter Reference - -| Parameter | Applies to | Description | -|-----------|-----------|-------------| -| `ref` | action | Element reference from snapshot | -| `action` | action | `click`, `type`, `fill`, `select`, `scroll` | -| `value` | type / fill / select | Text or option value | -| `direction` | scroll | `up`, `down`, `left`, `right` | -| `amount` | scroll | Scroll distance (lines or pixels) | -| `button` | click | `left`, `right`, `middle` | -| `clickCount` | click | `1` = single, `2` = double | -| `modifiers` | click | `shift`, `ctrl`, `meta` | -| `expression` | evaluate | JavaScript to execute | -| `tab_id` | Most commands | Target a specific tab | - -## Full Endpoint Map - -The `kuri-skill` CLI wraps all major Kuri HTTP endpoints. For the complete -list of ~80+ endpoints, see the main Kuri README. - -| Endpoint | `kuri-skill` | Purpose | -|----------|-------------|---------| -| `/health` | `health` | Server health check | -| `/tabs` | `tabs` | List all tabs | -| `/tab/new` | `tab-new` | Create a new tab | -| `/tab/close` | `action close-tab` | Close a tab | -| `/tab/current` | โ€” (use `KURI_SESSION`) | Set current tab | -| `/navigate` | `navigate` | Navigate to URL | -| `/page/info` | `page-info` | Page metadata | -| `/page/state` | โ€” | Lightweight observation | -| `/screenshot` | `screenshot` | Capture screenshot | -| `/snapshot` | `snap` | Accessibility tree | -| `/text` | `text` | Page plain text | -| `/markdown` | `markdown` | Page as markdown | -| `/links` | `links` | Extract links | -| `/action` | `action ` | Click, type, fill, select, scroll | -| `/evaluate` | `action evaluate` | Execute JavaScript | -| `/cookies` | `action cookies` | List cookies | -| `/audit` | `action audit` | Security audit | -| `/back` | `action back` | Browser back | -| `/forward` | `action forward` | Browser forward | -| `/reload` | `action reload` | Reload page | -| `/har` | `action har` | Current HAR log | -| `/har/start` | `action har-start` | Start HAR recording | -| `/har/stop` | `action har-stop` | Stop HAR recording | -| `/batch` | โ€” | Multi-command batch | -| `/evaluate` | `action evaluate` | JS evaluation | -| `/clipboard/*` | โ€” | Clipboard read/write | -| `/dialog/*` | โ€” | Dialog handling | -| `/mouse/*` | โ€” | Mouse control | -| `/recording/*` | โ€” | Action recording | -| `/vitals` | โ€” | Core Web Vitals | -| `/cache/*` | โ€” | Action caching | - -For the complete list see the main [Kuri README](../readme.md). diff --git a/skills/pi-kuri-plugin/references/ADVANCED.md b/skills/pi-kuri-plugin/references/ADVANCED.md new file mode 120000 index 0000000..fd286b7 --- /dev/null +++ b/skills/pi-kuri-plugin/references/ADVANCED.md @@ -0,0 +1 @@ +../../pi-kuri-advanced.md \ No newline at end of file diff --git a/skills/pi-kuri-plugin/scripts/kuri.js b/skills/pi-kuri-plugin/scripts/kuri.js index 1d4b8ea..4c1233a 100755 --- a/skills/pi-kuri-plugin/scripts/kuri.js +++ b/skills/pi-kuri-plugin/scripts/kuri.js @@ -29,9 +29,7 @@ * kuri-skill advanced # Print reference to ADVANCED.md */ -import { readFileSync, writeFileSync, existsSync, mkdirSync } from "node:fs"; -import { createWriteStream } from "node:fs"; -import { get } from "node:https"; +import { readFileSync, writeFileSync, existsSync } from "node:fs"; import { homedir } from "node:os"; import { resolve, dirname } from "node:path"; import { fileURLToPath } from "node:url"; @@ -78,7 +76,7 @@ function headers(session) { function isKuriInstalled() { try { - execSync("which kuri", { stdio: "ignore" }); + execSync("command -v kuri", { stdio: "ignore" }); return true; } catch { return false; @@ -247,7 +245,7 @@ async function cmdTabs() { async function cmdTabNew(url) { const params = {}; - if (url) params.url = encodeURI(url); + if (url) params.url = url; const data = await kuriFetch("/tab/new", params); const tabId = data?.result?.tabId || data?.result?.id || data?.id || ""; if (tabId) { @@ -298,20 +296,8 @@ async function resolveTabId(requested) { async function cmdPageInfo(tabId) { const params = {}; if (tabId) params.tab_id = tabId; - // Kuri v0.4.1+ may not have /page-info; fall back to /tabs filtered by tab - try { - const data = await kuriFetch("/page-info", params); - printJson(data); - } catch (err) { - // Fallback: list tabs and find the requested one - const tabs = await kuriFetch("/tabs"); - const list = Array.isArray(tabs) ? tabs : (tabs.tabs || tabs.result || []); - if (tabId) { - const tab = list.find((t) => t.id === tabId); - if (tab) { - console.log(JSON.stringify({ url: tab.url, title: tab.title, current: tab.current || false }, null, 2)); - return; - } + const data = await kuriFetch("/page/info", params); + printJson(data); } // Show current tab info const current = list.find((t) => t.current); @@ -588,15 +574,29 @@ Utilities: // Parse remaining args: action [key=value...] const sub = args[0]; const actionArgs = {}; + const actionSchemas = { + // Schema: [paramName, ...] for positional filling + // null means all remaining become expression + click: ["ref"], + type: ["ref", "value"], + fill: ["ref", "value"], + select: ["ref", "value"], + scroll: ["direction", "amount"], + evaluate: null, + "close-tab": ["tab_id"], + }; for (let i = 1; i < args.length; i++) { if (args[i].includes("=")) { const [k, ...v] = args[i].split("="); actionArgs[k] = v.join("="); - } else if (!actionArgs.ref && !["back", "forward", "reload"].includes(sub)) { - // Positional args fill in as needed - if (!actionArgs.ref) actionArgs.ref = args[i]; - else if (!actionArgs.value) actionArgs.value = args[i]; - else if (!actionArgs.direction) actionArgs.direction = args[i]; + } else if (actionSchemas[sub]) { + // Positional: fill schema slots in order + const slot = actionSchemas[sub][Object.keys(actionArgs).length]; + if (slot) actionArgs[slot] = args[i]; + } else if (actionSchemas[sub] === null) { + // evaluate: join all remaining as expression + actionArgs.expression = args.slice(1).join(" "); + break; } } await cmdAction(sub, actionArgs); From 8394d3628974223623fb34de2c529dec33855465 Mon Sep 17 00:00:00 2001 From: Sandcastle Smoke Test Date: Mon, 1 Jun 2026 14:29:47 -0400 Subject: [PATCH 7/8] =?UTF-8?q?docs(plugin):=20add=20OWNER.md=20=E2=80=94?= =?UTF-8?q?=20maintainer's=20guide=20for=20publishing=20to=20pi.dev?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Covers: - Directory layout and file purposes - How pi.dev auto-discovers packages from npm - Step-by-step publish workflow (version bump, smoke test, npm publish) - Maintaining skill docs in sync (symlink approach) - Version convention and real-world pi.dev package examples - Troubleshooting common issues --- skills/pi-kuri-plugin/OWNER.md | 225 +++++++++++++++++++++++++++++++++ 1 file changed, 225 insertions(+) create mode 100644 skills/pi-kuri-plugin/OWNER.md diff --git a/skills/pi-kuri-plugin/OWNER.md b/skills/pi-kuri-plugin/OWNER.md new file mode 100644 index 0000000..71f5945 --- /dev/null +++ b/skills/pi-kuri-plugin/OWNER.md @@ -0,0 +1,225 @@ +# pi-kuri-skill โ€” Maintainer's Guide + +This document covers how to publish and maintain `pi-kuri-skill` as a +[pi.dev](https://pi.dev) skill package on npm. + +## Quick Reference + +| What | Where | +|------|-------| +| Package directory | `skills/pi-kuri-plugin/` | +| npm package name | `pi-kuri-skill` | +| Install command | `pi install npm:pi-kuri-skill` | +| pi.dev catalog | [pi.dev/packages](https://pi.dev/packages) (auto-discovered from npm) | +| Example packages | `pi install npm:pi-subagents`, `pi install npm:pi-web-access` | + +## Directory Layout + +``` +skills/pi-kuri-plugin/ +โ”œโ”€โ”€ package.json # npm manifest โ€” name, version, pi manifest, bin +โ”œโ”€โ”€ pi-kuri.ts # Pi extension โ€” registers all Kuri tools for agents +โ”œโ”€โ”€ SKILL.md # Pi agent skill (two-tier: core + advanced) +โ”œโ”€โ”€ README.md # User-facing install/config docs +โ”œโ”€โ”€ scripts/ +โ”‚ โ””โ”€โ”€ kuri.js # CLI binary โ€” kuri-skill command +โ”œโ”€โ”€ references/ +โ”‚ โ””โ”€โ”€ ADVANCED.md โ†’ # Symlink โ†’ ../../pi-kuri-advanced.md (canonical source) +โ””โ”€โ”€ .npmignore # Excludes package-lock.json, node_modules +``` + +## How Publishing Works + +pi.dev auto-discovers packages from the npm registry. When you publish (or +update) an npm package with the `pi-package` keyword and a `pi` manifest in +`package.json`, it appears on the [pi.dev package catalog](https://pi.dev/packages) +within minutes. No separate submission is needed. + +### What pi.dev looks for in `package.json` + +```json +{ + "name": "pi-kuri-skill", + "version": "1.0.0", + "keywords": ["pi-package", "agent-skills", "kuri", "browser-automation"], + "pi": { + "skills": ["."] โ† tells pi.dev to find SKILL.md in package root + } +} +``` + +The `pi-package` keyword is the **discovery signal**. Without it, the package +won't appear on the catalog. + +For gallery previews, you can add optional metadata: + +```json +{ + "pi": { + "skills": ["."], + "image": "https://kuri.gg/screenshot.png", + "video": "https://kuri.gg/demo.mp4" + } +} +``` + +## Step-by-Step: Publish a Release + +### 1. Bump the version + +```bash +cd skills/pi-kuri-plugin + +# Choose one: +npm version patch # 1.0.0 โ†’ 1.0.1 (bug fixes) +npm version minor # 1.0.0 โ†’ 1.1.0 (new features, backward-compatible) +npm version major # 1.0.0 โ†’ 2.0.0 (breaking changes) +``` + +This updates `package.json`, creates a git tag, and commits. + +### 2. Smoke-test the package + +```bash +# Verify the CLI works +node scripts/kuri.js health + +# Verify the package tarball is clean +npm pack --dry-run +``` + +Check that `npm pack --dry-run` lists only the files you intend to ship. +The `.npmignore` file excludes `package-lock.json` and `node_modules`. + +### 3. Push the tag + +```bash +git push --tags +``` + +### 4. Publish to npm + +```bash +cd skills/pi-kuri-plugin +npm publish +``` + +> **Note**: You need npm publish access for the `pi-kuri-skill` package name. +> If you haven't published before, run `npm login` first and ensure the name +> is available (or use a scoped name like `@your-org/pi-kuri-skill`). + +### 5. Verify on pi.dev + +```bash +# Install from npm in a test project +pi install npm:pi-kuri-skill + +# Check it's listed +pi list +``` + +If the package is listed, it will appear on [pi.dev/packages](https://pi.dev/packages) +within a few minutes. + +## Maintaining the Skill + +### Keeping docs in sync + +The two-tier skill documentation lives at two levels: + +| File | Purpose | +|------|---------| +| `skills/pi-kuri-skills.md` | Canonical core skill docs (repo-level) | +| `skills/pi-kuri-advanced.md` | Canonical advanced reference (repo-level) | +| `skills/pi-kuri-plugin/SKILL.md` | Plugin skill entrypoint โ€” references both tiers | +| `skills/pi-kuri-plugin/references/ADVANCED.md` | **Symlink** โ†’ `../../pi-kuri-advanced.md` โ€” always in sync | + +The symlink at `references/ADVANCED.md` ensures the npm package always ships +the same advanced reference as the repo-level canonical file. **Do not replace +it with a copy** โ€” that's how drift happens. + +### Updating the Pi extension + +The extension file `pi-kuri.ts` registers Kuri tools for the Pi agent. When +you add, remove, or change tool signatures: + +1. Update `pi-kuri.ts` +2. Update `SKILL.md` if the user-facing instructions change +3. Bump a patch version and publish + +### Version convention + +| Change | Version bump | Example | +|--------|-------------|---------| +| Bug fix, docs, tool description | `patch` | 1.0.0 โ†’ 1.0.1 | +| New tool, new parameter, backward-compatible | `minor` | 1.0.0 โ†’ 1.1.0 | +| Breaking tool API change, removed tool | `major` | 1.0.0 โ†’ 2.0.0 | + +## Example: Other pi.dev Packages for Reference + +Browse the [pi.dev package catalog](https://pi.dev/packages) for real examples. +Notable packages with similar structure (skill + CLI/extension): + +| Package | Type | What to learn | +|---------|------|---------------| +| `pi-subagents` | extension | Multi-file extension with TUI, well-structured `package.json` | +| `pi-web-access` | extension | Tool registration pattern, multi-tool extension | +| `pi-mcp-adapter` | extension | Extension with MCP protocol integration | +| `pi-package-search` | skill | Skill-only package, `/skill:` invocation pattern | +| `@juicesharp/rpiv-*` | extension | Scoped package + comprehensive test suite | + +Install any of them to inspect their structure: + +```bash +pi install npm:pi-subagents +pi install npm:pi-web-access +pi install npm:pi-mcp-adapter +pi install npm:pi-package-search + +# Then inspect: +cat ~/.pi/agent/npm/node_modules/pi-subagents/package.json +``` + +## Troubleshooting + +### Package not appearing on pi.dev + +1. Check that `keywords` in `package.json` includes `"pi-package"` +2. Verify the `pi` manifest section is present +3. Confirm the package is publicly accessible on npm +4. Wait a few minutes โ€” the catalog refreshes periodically + +### `pi install npm:pi-kuri-skill` fails + +```bash +# Try with explicit version +pi install npm:pi-kuri-skill@1.0.0 + +# Check npm for the package +npm view pi-kuri-skill + +# Install from local path for testing +pi install ./skills/pi-kuri-plugin +``` + +### Symlink broken after clone + +The `references/ADVANCED.md` symlink targets `../../pi-kuri-advanced.md` +relative to the plugin directory. On a fresh clone, verify: + +```bash +ls -la skills/pi-kuri-plugin/references/ADVANCED.md +readlink -f skills/pi-kuri-plugin/references/ADVANCED.md +# Should resolve to: /path/to/kuri/skills/pi-kuri-advanced.md +``` + +If the symlink is broken, re-create it: + +```bash +rm skills/pi-kuri-plugin/references/ADVANCED.md +ln -s ../../pi-kuri-advanced.md skills/pi-kuri-plugin/references/ADVANCED.md +``` + +> **For Windows users**: Git may not handle symlinks by default. Enable them +> with `git config --global core.symlinks true` before cloning, or replace +> the symlink with a copy script in a prepublish step. From 1a3be81c9fadf9e60fa5b52485ac092761770af9 Mon Sep 17 00:00:00 2001 From: Sandcastle Smoke Test Date: Mon, 1 Jun 2026 14:33:10 -0400 Subject: [PATCH 8/8] fix(plugin): pi install handles extension registration, no manual symlink needed - Add "extensions": ["./pi-kuri.ts"] to package.json pi manifest so pi install auto-registers the Kuri tools alongside the skill. - Remove manual ln -s instructions from README.md, SKILL.md, and OWNER.md. - Update OWNER.md to document the auto-registration flow. --- skills/pi-kuri-plugin/OWNER.md | 11 ++++++++++- skills/pi-kuri-plugin/README.md | 10 +++------- skills/pi-kuri-plugin/SKILL.md | 10 +++------- skills/pi-kuri-plugin/package.json | 1 + 4 files changed, 17 insertions(+), 15 deletions(-) diff --git a/skills/pi-kuri-plugin/OWNER.md b/skills/pi-kuri-plugin/OWNER.md index 71f5945..8acd4e6 100644 --- a/skills/pi-kuri-plugin/OWNER.md +++ b/skills/pi-kuri-plugin/OWNER.md @@ -43,7 +43,8 @@ within minutes. No separate submission is needed. "version": "1.0.0", "keywords": ["pi-package", "agent-skills", "kuri", "browser-automation"], "pi": { - "skills": ["."] โ† tells pi.dev to find SKILL.md in package root + "extensions": ["./pi-kuri.ts"], โ† registers Kuri tools + "skills": ["."] โ† registers SKILL.md } } ``` @@ -56,6 +57,7 @@ For gallery previews, you can add optional metadata: ```json { "pi": { + "extensions": ["./pi-kuri.ts"], "skills": ["."], "image": "https://kuri.gg/screenshot.png", "video": "https://kuri.gg/demo.mp4" @@ -63,6 +65,13 @@ For gallery previews, you can add optional metadata: } ``` +### Extension auto-registration + +Because `package.json` declares `"extensions": ["./pi-kuri.ts"]`, running +`pi install npm:pi-kuri-skill` automatically registers all Kuri tools +(`kuri_navigate`, `kuri_snap`, `kuri_screenshot`, etc.) โ€” no manual symlink +or copy step needed. + ## Step-by-Step: Publish a Release ### 1. Bump the version diff --git a/skills/pi-kuri-plugin/README.md b/skills/pi-kuri-plugin/README.md index 9ac51de..336db05 100644 --- a/skills/pi-kuri-plugin/README.md +++ b/skills/pi-kuri-plugin/README.md @@ -33,13 +33,9 @@ npm install -g pi-kuri-skill ### Extension setup -To make the Pi extension available to your agent: - -```bash -ln -s $(pwd)/pi-kuri.ts ~/.pi/agent/extensions/pi-kuri.ts -``` - -Then restart pi.dev. The agent will have all Kuri tools registered. +That's it. `pi install` handles everything โ€” it registers the extension (tools like +`kuri_navigate`, `kuri_snap`, `kuri_screenshot`, etc.) and the skill (`SKILL.md`) +in one step. Restart pi.dev and the agent will have all Kuri tools available. Then use the `kuri-skill` command: diff --git a/skills/pi-kuri-plugin/SKILL.md b/skills/pi-kuri-plugin/SKILL.md index df5e5b1..f569651 100644 --- a/skills/pi-kuri-plugin/SKILL.md +++ b/skills/pi-kuri-plugin/SKILL.md @@ -23,13 +23,9 @@ manual use, debugging, and scripting. ### Install the Extension -Symlink `pi-kuri.ts` into your pi.dev agent extensions directory: - -```bash -ln -s $(pwd)/pi-kuri.ts ~/.pi/agent/extensions/pi-kuri.ts -``` - -After restarting pi.dev, the agent will have all Kuri tools available. +The package registers itself. `pi install npm:pi-kuri-skill` handles the +extension โ€” no manual symlink needed. Restart pi.dev and the agent will +have all Kuri tools available. ### Getting `kuri-skill` on PATH diff --git a/skills/pi-kuri-plugin/package.json b/skills/pi-kuri-plugin/package.json index 38fc72b..69218f7 100644 --- a/skills/pi-kuri-plugin/package.json +++ b/skills/pi-kuri-plugin/package.json @@ -8,6 +8,7 @@ "kuri-skill": "./scripts/kuri.js" }, "pi": { + "extensions": ["./pi-kuri.ts"], "skills": ["."] }, "scripts": {