Skip to content

add fullscreen touch interactions and clickable links - #2430

Merged
kevinjosethomas merged 5 commits into
mainfrom
eng-6201-touch-features
Sep 17, 2026
Merged

kevinjosethomas merged 5 commits into
mainfrom
eng-6201-touch-features

Conversation

@kevinjosethomas

@kevinjosethomas kevinjosethomas commented Sep 16, 2026

Copy link
Copy Markdown
Member
  • adds fullscreen pointer and touch support for cursor placement and per-item expansion for eng-6201.
  • makes labeled markdown links clickable while retaining visible url fallbacks for eng-6126.
  • preserves global detail cycling, text selection, hyperlink precedence, and inline terminal behavior.

Note

Medium Risk
Large surface area in fullscreen mouse dispatch, editor hit-testing, and many chat components, though behavior is covered by new tests and existing keyboard/selection paths are preserved.

Overview
Adds render-time click regions to the TUI: components expose regions after render, containers aggregate them, and the fullscreen viewport projects them through the pinned header, scrolling transcript, clipped dock, and overlays. A clean unmodified left press/release dispatches clicks (hyperlinks and selection/drag/modifier clicks keep precedence; painted overlay pixels block click-through).

Editor clicks focus the input and place the cursor from the last render layout (wrapping, scroll, prompt/header offsets, wide graphemes, atomic paste markers). Markdown labeled links stay clickable even when a visible URL fallback is shown.

In the coding-agent interactive UI, collapsed headers and summary rows (tools, bash, agent messages, branch/compaction/refinement summaries, skills, custom/injected prompts, collapsible errors, shell blocks in side questions, etc.) toggle only that item via Clickable or explicit regions; Ctrl+O still resets all conversation detail globally.

Reviewed by Cursor Bugbot for commit 7daebc3. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Add fullscreen click dispatch, editor click-to-cursor, and clickable Markdown links

  • Core click-region system in click-regions.ts, fullscreen.ts, and tui.ts: components expose getClickRegions after render; the fullscreen viewport projects regions through header, transcript, and dock layout and resolves unmodified left-button press/release pairs into region callbacks.
  • Clickable wrapper in clickable.ts registers a click region over any child's full rendered output.
  • Editor clicks in editor.ts map through saved render layout to source-line cursor positions, handling wrapping, Unicode, prompt prefixes, and atomic paste markers.
  • Markdown link labels in markdown.ts now emit hyperlink click metadata in both native and fallback rendering paths.
  • Interactive chat components (agent messages, bash/tool execution, branch summaries, collapsible errors, and others) use click regions to toggle expanded/collapsed state on header clicks; Ctrl+O still resets all expansions globally.
  • Risk: tui.Component interface gains optional getClickRegions; out-of-tree components without it render normally but cannot dispatch clicks. Container and Box click-region caches are invalidated on child add/remove/clear and on render-cache invalidation to prevent stale regions.

Changes since #2430 opened

  • Made the summary row of ExpandableEventMessage-derived components clickable to toggle expanded state [7daebc3]
  • Updated test needle for locating compaction row click region [7daebc3]

Macroscope summarized d24970d.

@github-actions

github-actions Bot commented Sep 16, 2026

Copy link
Copy Markdown

Prime Agent performance — completed

PR 7daebc36 compared with main 7572880f.

Overall: 0 regressed · 0 improved · 41 no clear change.

Metric Main This PR Change
Cold startup 1,749.0 ms 1,826.1 ms ≈ +77.1 ms (+4.41%)
Warm startup 1,109.7 ms 1,004.5 ms ≈ -105.2 ms (-9.48%)
Installation 14.21 s 14.84 s ≈ +0.63 s (+4.42%)
Compressed release artifacts 72.51 MB 72.54 MB ≈ +0.02 MB (+0.03%)
Installed footprint 577.65 MB 577.72 MB ≈ +0.07 MB (+0.01%)
Idle memory, summed RSS 1,272.13 MB 1,271.29 MB ≈ -0.84 MB (-0.07%)

Python runtime

Metric Main This PR Change
Python kernel startup 152.4 ms 163.3 ms ≈ +10.8 ms (+7.11%)
Python cell round trip 0.577 ms 0.640 ms ≈ +0.063 ms (+10.93%)
Empty bash command 13.4 ms 14.7 ms ≈ +1.3 ms (+9.65%)
Bash git status 20.0 ms 20.6 ms ≈ +0.6 ms (+3.10%)
Bash 32 KiB output 13.7 ms 14.8 ms ≈ +1.1 ms (+7.77%)
35 cells / 9 shell calls 197.0 ms 196.0 ms ≈ -1.0 ms (-0.52%)
Python interrupt to done 1.712 ms 1.698 ms ≈ -0.014 ms (-0.81%)
Python state snapshot 27.2 ms 29.0 ms ≈ +1.8 ms (+6.76%)
Python state restore 389.6 ms 402.1 ms ≈ +12.5 ms (+3.22%)
Python idle RSS 35.67 MB 31.83 MB ≈ -3.84 MB (-10.76%)
Python RSS after pandas workload 98.09 MB 94.92 MB ≈ -3.17 MB (-3.24%)

Session transport

Metric Main This PR Change
Private frame decode, 32 MiB in 8 KiB chunks 32.7 ms 36.2 ms ≈ +3.4 ms (+10.47%)

UI interactions

Metric Main This PR Change
Resume large session (cold) 4,459.2 ms 4,256.4 ms ≈ -202.8 ms (-4.55%)
CPU, resume large session 7,740.0 ms 7,110.0 ms ≈ -630.0 ms (-8.14%)
Switch into large session 4,956.3 ms 4,238.3 ms ≈ -718.0 ms (-14.49%)
CPU, switch into large session 6,880.0 ms 6,280.0 ms ≈ -600.0 ms (-8.72%)
Open agents view from a session 191.9 ms 178.8 ms ≈ -13.2 ms (-6.86%)
CPU, open agents view 400.0 ms 350.0 ms ≈ -50.0 ms (-12.50%)
Full agents roster, many sessions 4.86 s 4.86 s ≈ +0.00014 s (+0.00%)
CPU, full agents roster 3.44 s 3.16 s ≈ -0.28 s (-8.14%)
Open another session from agents view 2,991.4 ms 2,766.5 ms ≈ -224.8 ms (-7.52%)
CPU, open from agents view 3,200.0 ms 2,840.0 ms ≈ -360.0 ms (-11.25%)
Reopen resident large session 601.9 ms 669.6 ms ≈ +67.7 ms (+11.24%)
CPU, reopen resident session 1,000.0 ms 1,170.0 ms ≈ +170.0 ms (+17.00%)
Open subagent session at depth 6 21,715.2 ms 21,761.9 ms ≈ +46.7 ms (+0.21%)
CPU, open subagent at depth 6 11,540.0 ms 11,180.0 ms ≈ -360.0 ms (-3.12%)
Open chain parent from agents view 4,127.9 ms 4,145.0 ms ≈ +17.2 ms (+0.42%)
CPU, open chain parent 4,120.0 ms 4,710.0 ms ≈ +590.0 ms (+14.32%)
Scheduled catalog, first request 2,341.9 ms 2,239.8 ms ≈ -102.2 ms (-4.36%)
CPU, scheduled catalog 3,910.0 ms 3,910.0 ms ≈ -4.4e-13 ms (-0.00%)
Scheduled catalog, repeated request 1,264.3 ms 1,518.5 ms ≈ +254.2 ms (+20.11%)
CPU, repeated catalog 1,730.0 ms 2,070.0 ms ≈ +340.0 ms (+19.65%)
Cold worker with three catalog scans 1,665.8 ms 1,769.9 ms ≈ +104.1 ms (+6.25%)
CPU, cold worker and scans 4,860.0 ms 4,830.0 ms ≈ -30.0 ms (-0.62%)
UI memory after interactions 2,693.92 MB 2,663.25 MB ≈ -30.67 MB (-1.14%)

Sandbox cost: ~$0.1653 — no inference calls.
Run, logs, and downloadable raw results

Methodology and samples

Main resolved at 2026-09-17T21:49:15.594096+00:00. Harness 7572880f.
Linux x64, 4 vCPU, 8 GB RAM, 20 GB disk; region us.
Image: node:24-bookworm@sha256:be23f54a88d34e8824c741b19b91064094f92c1c97b194144bfc8b50d67258e2.
Stock tools, skills, daemon, and Python bootstrap enabled; fresh homes and a fixed Git fixture.
Onboarding is dismissed; the editor starts without a selected model or submitted prompt.
Medians shown. Arrows require a 20% timing/memory change plus absolute floors and IQR.
These practical noise floors are not a statistical significance test.
Cold means stopped Prime processes; OS filesystem caches are not flushed.
No model requests or credentials. Installation excludes build/setup time.
Installer tarballs use loopback; npm/Python downloads use the network with fresh caches.
Artifact size counts release tarballs; footprint after first use includes registry packages.
MB is decimal. Summed RSS can double-count shared pages; PSS is recorded when available.
Provisioning, setup, and build durations are recorded separately in the raw results.
Kernel probes use the installed JSONL runtime, outside the TUI/TypeScript host.
Per trial: 50 Python cells, 5 calls per shell case, and one 35-cell mix (9 git status calls).
Cell/shell values are batch means; other runtime timings are single operations.
State fixture: a 10,000-row × 8-column integer DataFrame and a 10,000-integer list.
Restore runs in a fresh kernel, including pandas imports; kernel startup is excluded.
Kernel RSS covers the isolated Python process; loaded RSS follows the pandas workload.
Transport benches run node against the prepared source build, outside the installed home.
Frame decode times one 32 MiB private frame, snapshot-chunk header, pushed in
8 KiB chunks; the wire shape of multi-MB frames on the daemon-worker channels.
UI trials use a fresh fixture set: 194 top-level sessions including one ~40 MB transcript,
40 ledger fan-out children, and a 6-deep subagent chain (~46 spawn edges).
Large fixtures hold 1,999 complete triples (~5 MB JSONL); medium 119; subagents 399 each.
Interactions: cold --resume of a large session, warm /resume switch, left-arrow to agents view,
roster settle with many saved sessions, search-and-open of another large session,
reattaching to that resident session, opening the chain parent, and drilling to depth 6.
Readiness is the rendered transcript tail plus a confirmed editor echo.
CPU metrics sum utime+stime across the whole benchmark-user process tree per interaction.
UI memory sums RSS after the interactions; PTY byte counts are in the raw results.
A separate catalog fixture has 2,300 sessions, 2,298 edges, and 13 paused scheduled-job owners.
Catalog timings cover first/repeated reads and cold worker creation under three pending scans.
All expected jobs and owner metadata are checked; worker readiness excludes TUI rendering.
Costs estimate full sandbox lifetimes at configured rates, including setup and build.
Budget target: $1; not a billing cap. Performance changes are informational.
Failed or incomplete execution fails the workflow; saved artifacts remain available.
Each side stops a phase after 2 identical consecutive failures.
Skipped trials are not attempted samples. Warm startup requires a successful cold launch.

Metric Main successful/attempted PR successful/attempted Main spread PR spread
Cold startup 10/10 10/10 IQR 140.1 ms IQR 163.1 ms
Warm startup 10/10 10/10 IQR 144.3 ms IQR 42.7 ms
Installation 3/3 3/3 range 0.44 s range 0.31 s
Compressed release artifacts 1/1 1/1
Installed footprint 1/1 1/1
Idle memory, summed RSS 10/10 10/10 IQR 15.40 MB IQR 15.91 MB
Python kernel startup 10/10 10/10 IQR 20.4 ms IQR 12.9 ms
Python cell round trip 10/10 10/10 IQR 0.086 ms IQR 0.118 ms
Empty bash command 10/10 10/10 IQR 2.3 ms IQR 3.4 ms
Bash git status 10/10 10/10 IQR 3.2 ms IQR 5.1 ms
Bash 32 KiB output 10/10 10/10 IQR 3.8 ms IQR 4.1 ms
35 cells / 9 shell calls 10/10 10/10 IQR 18.5 ms IQR 33.7 ms
Python interrupt to done 10/10 10/10 IQR 0.115 ms IQR 0.150 ms
Python state snapshot 10/10 10/10 IQR 2.8 ms IQR 10.6 ms
Python state restore 10/10 10/10 IQR 42.0 ms IQR 53.5 ms
Python idle RSS 10/10 10/10 IQR 4.22 MB IQR 3.18 MB
Python RSS after pandas workload 10/10 10/10 IQR 3.90 MB IQR 3.77 MB
Private frame decode, 32 MiB in 8 KiB chunks 10/10 10/10 IQR 3.8 ms IQR 3.5 ms
Resume large session (cold) 3/3 3/3 range 1,178.1 ms range 919.9 ms
CPU, resume large session 3/3 3/3 range 1,760.0 ms range 1,670.0 ms
Switch into large session 3/3 3/3 range 1,151.5 ms range 217.6 ms
CPU, switch into large session 3/3 3/3 range 980.0 ms range 870.0 ms
Open agents view from a session 3/3 3/3 range 16.8 ms range 16.6 ms
CPU, open agents view 3/3 3/3 range 420.0 ms range 180.0 ms
Full agents roster, many sessions 3/3 3/3 range 0.41 s range 0.41 s
CPU, full agents roster 3/3 3/3 range 0.67 s range 0.72 s
Open another session from agents view 3/3 3/3 range 381.2 ms range 204.6 ms
CPU, open from agents view 3/3 3/3 range 790.0 ms range 360.0 ms
Reopen resident large session 3/3 3/3 range 131.9 ms range 99.4 ms
CPU, reopen resident session 3/3 3/3 range 400.0 ms range 340.0 ms
Open subagent session at depth 6 3/3 3/3 range 1,469.8 ms range 301.0 ms
CPU, open subagent at depth 6 3/3 3/3 range 2,600.0 ms range 430.0 ms
Open chain parent from agents view 3/3 3/3 range 359.1 ms range 179.2 ms
CPU, open chain parent 3/3 3/3 range 670.0 ms range 290.0 ms
Scheduled catalog, first request 3/3 3/3 range 535.9 ms range 169.0 ms
CPU, scheduled catalog 3/3 3/3 range 960.0 ms range 550.0 ms
Scheduled catalog, repeated request 3/3 3/3 range 316.8 ms range 170.9 ms
CPU, repeated catalog 3/3 3/3 range 450.0 ms range 260.0 ms
Cold worker with three catalog scans 3/3 3/3 range 339.3 ms range 360.5 ms
CPU, cold worker and scans 3/3 3/3 range 940.0 ms range 940.0 ms
UI memory after interactions 3/3 3/3 range 73.74 MB range 80.55 MB

Comment thread packages/coding-agent/src/modes/interactive/components/agent-message.ts Outdated

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@sethkarten sethkarten closed this Sep 17, 2026
@sethkarten
sethkarten deleted the eng-6201-touch-features branch September 17, 2026 07:01
@kevinjosethomas
kevinjosethomas restored the eng-6201-touch-features branch September 17, 2026 17:45
@sethkarten
sethkarten self-requested a review September 17, 2026 21:24
sethkarten
sethkarten previously approved these changes Sep 17, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit d24970d. Configure here.

@kevinjosethomas
kevinjosethomas merged commit b6fb298 into main Sep 17, 2026
56 checks passed
@kevinjosethomas
kevinjosethomas deleted the eng-6201-touch-features branch September 17, 2026 22:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants