Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
c73923a
docs for CDP layer plan
Dingway98 Nov 10, 2025
19d111a
phase 1.1 workstream A
Dingway98 Nov 11, 2025
40c1b29
finished up Phase 1
Dingway98 Nov 11, 2025
05781f7
phase 2 - initial working version
Dingway98 Nov 11, 2025
acf3141
fix edge cases for CDP action interactions
Dingway98 Nov 12, 2025
f033bd4
phase 2 polish
Dingway98 Nov 12, 2025
7ae18e5
pahse 3 docs plan
Dingway98 Nov 12, 2025
5b9ab38
initial working version of CDP first frames
Dingway98 Nov 12, 2025
0dcfeb9
auto attach frames
Dingway98 Nov 12, 2025
dcdb63f
finish phase 3 with some leftover code
Dingway98 Nov 13, 2025
5f2b882
optimized dom settling and CDP session management
Dingway98 Nov 13, 2025
b0b94e5
more optimizations
Dingway98 Nov 14, 2025
ed1af7d
remove planning docs
Dingway98 Nov 14, 2025
b2f3878
make geta11ydom retry if couldn't get AX TRee
Dingway98 Nov 14, 2025
d71fdfd
fix scroll, removed feature flags
Dingway98 Nov 14, 2025
f841c97
update .gitignore
Dingway98 Nov 14, 2025
1089613
simplify CDP opt out
Dingway98 Nov 14, 2025
ff7dcbf
clean up debug logging
Dingway98 Nov 14, 2025
0857000
update readme
Dingway98 Nov 14, 2025
0c0b086
remove unused deps
Dingway98 Nov 14, 2025
628f338
remove more logging
Dingway98 Nov 14, 2025
78df8f3
fix
Dingway98 Nov 14, 2025
7ce1533
fix
Dingway98 Nov 14, 2025
3d3e97c
fix oopif
Dingway98 Nov 15, 2025
ff979d7
fix frame mapping, remove legacy code
Dingway98 Nov 16, 2025
a7eab81
fix screenshots
Dingway98 Nov 16, 2025
f87fd85
tighten the prompt some more
Dingway98 Nov 16, 2025
676d673
update scripts
Dingway98 Nov 16, 2025
7ac6532
add more comments and DOM pierce: false for OOPIF
Dingway98 Nov 17, 2025
fb21bdf
fix resolve
Dingway98 Nov 17, 2025
08bbf11
parallelize iframe fetching
Dingway98 Nov 17, 2025
02e05a1
captureOOPIF on every dom refresh
Dingway98 Nov 17, 2025
0c25905
cache CDP OOPIF sessions
Dingway98 Nov 17, 2025
d64996b
cache CDP OOPIF frames, fix prompt, use tools for claude
Dingway98 Nov 18, 2025
e12460a
ignore ads. fix prompt. make agent better when it's long task
Dingway98 Nov 18, 2025
40e1c8c
add errors from schema, so LLM learns from them
Dingway98 Nov 18, 2025
225b039
update docs
Dingway98 Nov 18, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ yarn-error.log*
lerna-debug.log*
.pnpm-debug.log*
tmp
debug
/debug

# Diagnostic reports (https://nodejs.org/api/report.html)
report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json
Expand Down
31 changes: 28 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ HyperAgent provides two complementary APIs optimized for different use cases:
- ⚡ **Fast** - Uses accessibility tree (no screenshots)
- 💰 **Cheap** - Single LLM call per action
- 🎯 **Reliable** - Direct element finding and execution
- 📊 **Efficient** - Text-based DOM analysis
- 📊 **Efficient** - Text-based DOM analysis with automatic ad-frame filtering

**Example**:
```typescript
Expand All @@ -136,18 +136,27 @@ await page.aiAction("click the login button");
**Best for**: Complex workflows requiring multiple steps and visual context

**Advantages**:
- 🖼️ **Visual Understanding** - Uses screenshots with element overlays

- 🖼️ **Visual Understanding** - Can use screenshots with element overlays
- 🎭 **Complex Tasks** - Handles multi-step workflows automatically
- 🧠 **Context-Aware** - Better at understanding page layout and relationships
- 🔄 **Adaptive** - Can adjust strategy based on page state

**Parameters**:

- `useDomCache` (boolean): Reuse DOM snapshots for speed
- `enableVisualMode` (boolean): Enable screenshots and overlays (default: false)

**Example**:

```typescript
const page = await agent.newPage();
await page.goto("https://flights.google.com");

// Complex task with multiple steps handled automatically
await page.ai("search for flights from Miami to New Orleans on July 16");
await page.ai("search for flights from Miami to New Orleans on July 16", {
useDomCache: true,
});
```

### 🎨 Mix and Match
Expand Down Expand Up @@ -370,6 +379,22 @@ const agent = new HyperAgent({
});
```

## CDP First

HyperAgent speaks Chrome DevTools Protocol natively. Element lookup, scrolling, typing, frame management, and screenshots all go through CDP so every action has exact coordinates, execution contexts, and browser events. This allows for more custom commands and deep iframe tracking.

HyperAgent integrates seamlessly with Playwright, so you can still use familiar commands, while the actions take full advantage of native CDP protocol with fast locators and advanced iframe tracking.

**Key Features:**

- **Auto-Ad Filtering**: Automatically filters out ad and tracking iframes to keep context clean
- **Deep Iframe Support**: Tracking across nested and cross-origin iframes (OOPIFs)
- **Exact Coordinates**: Actions use precise CDP coordinates for reliability

Keep in mind that CDP is still experimental, and stability is not guaranteed. If you’d like the agent to use Playwright’s native locators/actions instead, set `cdpActions: false` when you create the agent and it will fall back automatically.

The CDP layer is still evolving—expect rapid polish (and the occasional sharp edge). If you hit something quirky you can toggle CDP off for that workflow and drop us a bug report.

## Contributing

We welcome contributions to Hyperagent! Here's how you can help:
Expand Down
203 changes: 0 additions & 203 deletions docs/MIGRATION.md

This file was deleted.

Loading