Skip to content
ZJU-REALPublic

Repository files navigation

CUA-JEV

Jev for Computer Use

Webpage · Watch demos · 中文文档

CUA-JEV: a robot at a computer picks one of five routes — GUI, DOM, CLI, MCP or File — for its next step, and a second robot verifies the result before the loop repeats

CUA-JEV is an open-source reference framework for computer-use agents built on Jev. A text model proposes grounded next actions; task adapters turn them into legal, executable, verifiable intent × channel candidates (GUI, DOM, UIA, COM, CLI, MCP, file API); Jev picks the action and the route that executes it; a guard, an executor and an independent verifier close the loop.

Recorded single runs, not benchmarks. Published desktop workflows are Windows-only; arbitrary-task generalization is not claimed.

How it works

  1. Observe — an adapter reads live state and lists only the currently legal intent × channel candidates.
  2. Select — Jev's choice API returns one candidate ID; the action and its arguments are already typed, so Jev writes no code.
  3. Guard & execute — ActionGuard checks freshness, identity, scope and permissions, then a registered executor runs the route.
  4. Verify & repeat — an independent verifier checks the new state, a JSONL trace records the step, and the loop observes again.

Reusable pieces: ActionCandidate, AgentRuntime, ActionGuard, ExecutorRegistry.

Recorded Windows runs

Four real model + Jev runs (18–21 actions each, zero VLM calls). Click a preview for the video and step trace.

Python automation · 19 actions JavaScript study guide · 18 actions
Python automation guide recording preview JavaScript study guide recording preview
Git workflow · 21 actions PowerShell learning guide · 19 actions
Git workflow guide recording preview PowerShell learning guide recording preview

Earlier task-specific workflows, open-task pilots and the roadmap: docs/EXPERIMENTS.md · open-task notes.

Quick start

Windows, Python 3.11+. The keyless Rule baseline runs first:

py -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install -e ".[dev,all,ui]"
playwright install chromium
cua-jev doctor
cua-jev demo --policy rule

With Jev (keep the key in the Git-ignored .env):

Copy-Item .env.example .env   # set TYPESAFE_API_KEY=...
cua-jev jev-smoke
cua-jev suite --task vscode --policy jev --profile adaptive --open-vscode

--task edge|excel|vscode|explorer|all; --profile adaptive = Hybrid, visible = GUI only. Local showcase: cua-jev-ui → http://127.0.0.1:8768.

Extend with another task

Write an observer with a verifiable state (observers.py), emit an ActionCandidate for every legal intent × channel, register executors (registry.py), and add an independent verifier (verify.py). Start from inspect_report.json.

Safety, license, acknowledgements

ActionGuard fails closed on identity, stale observations, allowed roots, writes and external side effects; CLI uses registered argv templates only, and API keys never enter traces or the repo. Code under Apache-2.0. Thanks to TypeSafe AI for Jev and to RoboJEV for inspiration; CUA-JEV is an independent project, not an official TypeSafe AI product.

Releases

Packages

Contributors

Languages