Skip to content

紧急兼容新版 App 的桌宠真实鼠标交互 / Urgent compatibility fix for real-mouse pet interaction#1555

Open
Yuimi-chaya wants to merge 2 commits into
BigPizzaV3:mainfrom
Yuimi-chaya:codex/pet-real-mouse-app-compat
Open

紧急兼容新版 App 的桌宠真实鼠标交互 / Urgent compatibility fix for real-mouse pet interaction#1555
Yuimi-chaya wants to merge 2 commits into
BigPizzaV3:mainfrom
Yuimi-chaya:codex/pet-real-mouse-app-compat

Conversation

@Yuimi-chaya

@Yuimi-chaya Yuimi-chaya commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

中文

为什么需要这次紧急兼容更新

PR #1455 合并后,Codex++ 已经能够把 V2 桌宠原本由 Computer Use 虚拟光标驱动的视线动作映射到 Windows 真实鼠标。这个功能在当时的 Codex App 中可以正常工作,但 Codex App 26.715.2305.0 之后更新了 avatar overlay 的页面生命周期、模块加载时序和 execution context 切换方式,原有 runtime 开始出现新的竞态。

真实使用中会出现以下现象:

  • 桌宠只响应视线转向,但原生 hover 或拖拽偶尔失效;
  • 鼠标放到桌宠上时,点击可能落到桌宠后方的界面元素;
  • 刷新开关可以暂时恢复,但页面导航或 runtime 重建后仍可能复发;
  • 短暂的 CDP/能力探测失败会被误判为“不支持 V2”,随后旧清理逻辑可能停掉刚刚建立的新 runtime。

后续排查还确认了一个更直接的拖拽根因:原脚本只维护 dragging 布尔状态,没有真正建立浏览器 Pointer Capture。指针离开桌宠边界后,后续事件可能被下层页面接收,于是表现为拖不动、拖到一半失去交互,或点击直接穿透到桌宠背后的界面。

这不是 #1455 初始实现时就存在的问题,而是新版 Codex App 改变运行时行为后产生的兼容回归。如果不及时合并,已经进入主干的真实鼠标桌宠功能在新版 App 上会表现为时好时坏,用户只能反复切换开关或重启,并且可能遇到桌宠无法拖拽、点击穿透等明显交互问题。

修改内容

  • 将真实鼠标 runtime 升级到 v6,确保页面重建后不会继续复用旧 runtime。
  • dispatcher 动态导入和订阅完成前后都检查 runtime 是否已经退役;迟到的订阅会立即解除,不会污染替代 runtime。
  • 使用桌宠几何边界判断真实鼠标是否位于桌宠区域,优先释放合成视线点,让原生 hover 和拖拽接管交互。
  • 新增共享交互所有权锁,防止多个桌宠 runtime 同时争夺同一个拖拽指针。
  • 拖拽开始时调用 setPointerCapture,结束时调用 releasePointerCapture;即使指针移出桌宠边界,拖拽事件仍由桌宠持有。
  • pointercancellostpointercapture、窗口失焦、runtime 停止和重新注入时统一清理拖拽所有权,避免残留状态。
  • 拖拽期间暂停合成视线更新并阻止事件继续落向下层页面,同时保留几何命中兜底以适配新版 App 的透明命中变化。
  • 区分“明确不支持 V2”和“CDP/导航探测失败”。瞬时错误只上报,不再向可能已经切换 execution context 的页面发送 stop。
  • 某个 target driver 退出时不再批量停止全部 target;只有设置被正常关闭时,当前 driver 才执行 stop 清理。

范围边界

  • 仅修改真实鼠标注入脚本、launcher 生命周期和对应测试,共 3 个文件。
  • 不修改用户宠物素材。
  • 保留现有 V2 能力检测和 Windows 平台限制。
  • 本 PR 不包含桌宠持续状态、持续动画、气泡或 notification expiry patch。

验证

  • node --check assets/inject/pet-real-mouse-inject.js
  • cargo fmt --all -- --check
  • git diff --check
  • 原完整回归:cargo test -p codex-plus-core --test cdp_bridge -j 278/78 通过
  • Pointer Capture 追加合同测试:1/1 通过
  • cargo check -p codex-plus-launcher -j 2
  • Codex App 26.715.4045.0 真人测试通过:桌宠边界外持续拖拽、松开恢复、点击穿透防护、原生 hover 与真实鼠标视线优先级均正常

这是针对已合并功能的新版 App 紧急兼容更新,希望可以尽快审查并合并,避免 #1455 在当前 Codex App 上继续以不稳定状态提供给用户。


English

Why this urgent compatibility update is needed

After PR #1455 was merged, Codex++ could map the V2 pet gaze animation, which was originally driven by the Computer Use virtual cursor, to the real Windows mouse. The feature worked with the Codex App version available at that time. Codex App releases after 26.715.2305.0 changed the avatar-overlay page lifecycle, module-loading timing, and execution-context transitions, exposing new races in the existing runtime.

Observed symptoms include:

  • the pet follows the pointer direction, while native hover or drag intermittently stops working;
  • pointer input can reach UI elements behind the pet;
  • toggling the setting temporarily recovers the feature, but navigation or runtime recreation can trigger the issue again;
  • a transient CDP/capability-probe failure is treated as “V2 unsupported”, after which stale cleanup can stop the newly installed runtime.

Further investigation confirmed a direct drag root cause: the old script tracked only a dragging boolean and never established browser Pointer Capture. Once the pointer left the pet bounds, subsequent events could be delivered to the underlying page, causing drag loss or click-through interaction.

This was not an original #1455 defect. It is a compatibility regression caused by newer Codex App runtime behavior. Without this update, the already-merged real-mouse pet feature remains unreliable on current App versions, forcing users to toggle or restart and sometimes leaving the pet non-draggable with click-through interaction.

Changes

  • Bump the real-mouse runtime to v6 so a rebuilt page cannot retain the previous runtime.
  • Check runtime retirement before and after dynamic dispatcher subscription; immediately unsubscribe late listeners.
  • Use mascot geometry to yield synthetic gaze input whenever the real pointer is inside the pet region, preserving native hover and drag.
  • Add shared interaction ownership so multiple pet runtimes cannot claim the same drag pointer.
  • Call setPointerCapture on drag start and releasePointerCapture on completion, retaining drag ownership outside the pet bounds.
  • Clean up drag ownership on pointercancel, lostpointercapture, window blur, runtime shutdown, and reinjection.
  • Suspend synthetic gaze updates and suppress underlying-page input while dragging, with a geometry fallback for current transparent hit-testing behavior.
  • Distinguish explicit V2 incompatibility from transient CDP/navigation probe errors. Probe failures are reported without sending a stop command into a potentially replaced execution context.
  • Do not stop every target when one target driver exits. A target sends the stop script only after an intentional settings shutdown.

Scope

  • Three files only: the real-mouse injection runtime, launcher lifecycle, and regression assertions.
  • No user pet assets are modified.
  • Existing V2 capability detection and Windows gating remain intact.
  • Persistent pet status, persistent animation, bubbles, and notification-expiry patching are intentionally excluded.

Validation

  • node --check assets/inject/pet-real-mouse-inject.js
  • cargo fmt --all -- --check
  • git diff --check
  • Previous full regression: cargo test -p codex-plus-core --test cdp_bridge -j 2, 78/78 passed
  • Added Pointer Capture contract: 1/1 passed
  • cargo check -p codex-plus-launcher -j 2
  • Human-validated on Codex App 26.715.4045.0: dragging beyond pet bounds, release recovery, click-through prevention, native hover, and real-mouse gaze priority all passed

Because this repairs an already-merged feature broken by newer App lifecycle behavior, an expedited review and merge would prevent #1455 from remaining unreliable for current Codex App users.

中文:适配新版 Codex App 的 avatar overlay 生命周期变化,避免瞬时能力探测失败、旧 driver 清理和迟到 dispatcher 订阅停掉新的真实鼠标 runtime。改用宠物几何边界优先让出原生 hover/拖拽。此提交不包含桌宠持续状态功能。

English: Adapt the real-mouse pet runtime to the new Codex App avatar-overlay lifecycle. Transient capability probe failures, stale driver cleanup, and late dispatcher subscriptions no longer retire the replacement runtime. Mascot geometry yields to native hover and drag interaction. Persistent pet status is intentionally excluded.
@Yuimi-chaya
Yuimi-chaya force-pushed the codex/pet-real-mouse-app-compat branch from 858b35f to 29f8951 Compare July 18, 2026 23:43
@Yuimi-chaya

Copy link
Copy Markdown
Contributor Author

已补充 Pointer Capture 与共享拖拽所有权修复。进一步排查确认,旧实现仅维护 dragging 状态,指针移出桌宠边界后事件可能落到下层页面,导致偶发拖拽失效或点击穿透。当前追加补丁已在 Codex App 26.715.4045.0 完成真人测试,边界外持续拖拽、松开恢复、点击穿透防护以及 hover/视线优先级均通过。PR 文案与验证信息也已同步更新,麻烦继续审查。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant