Skip to content

fix(executor): keep the hot-queue walk alive across nested ticks - #1058

Closed
x-at-01 wants to merge 1 commit into
compio-rs:masterfrom
x-at-01:fix/executor-tick-reentrant
Closed

x-at-01 wants to merge 1 commit into
compio-rs:masterfrom
x-at-01:fix/executor-tick-reentrant

Conversation

@x-at-01

@x-at-01 x-at-01 commented Sep 23, 2026

Copy link
Copy Markdown

Problem

Executor::tick walks the hot queue with an iterator that lazily prefetches the successor via TaskQueue::next_hot, while the loop body has already moved the previously returned task off the hot list (make_cold also clears its next pointer).

Whenever a task's poll re-enters tick on the same executor — nested driving, which happens in practice with a synchronous harvest inside a thread-per-core runtime (calling the low-level run/poll pair from within a task's poll, in the spirit of a blocking-wait bridge) — the inner tick transitions the prefetched task from hot to cold. The outer iteration then:

  • panics on debug builds: debug_assert!(item.is_hot) in next_hot, or
  • silently truncates the walk on release builds (item.next is already None), leaving queued tasks unpolled until the next tick.

We hit this in production-shaped tests of our thread-per-core service: with multiple runtimes and frequent cross-thread wakes, workers died randomly on assertion failed: item.is_hot (compio-executor 0.1.4), while single-task runs were always green.

Fix

Handle the stale cursor explicitly instead of asserting:

  1. next_hot: when the cursor's task is no longer hot, restart from the current hot head.
  2. tick's loop body: skip tasks that a nested tick has already taken off the hot list (TaskQueue::is_hot), so a nested tick never causes a double poll.

Re-entry is now crash-free and lossless, and with no nesting the iteration behaves exactly as before (every queued task visited at most once per tick; tasks woken during the tick are picked up by a later tick, unchanged).

Testing

  • New regression test test_tick_is_reentrant_safe: task A re-enters tick from its poll while task B (never completing) is queued. On master this panics at debug_assert!(item.is_hot) (verified by reverting the src changes with the test in place); with the fix it passes and B simply rests in the cold queue.
  • Full compio-executor suite green (8 passed), including the existing cancellation tests, confirming no behavioral change for the non-nested case.

@github-actions github-actions Bot added bug Something isn't working package: executor Related to compio-executor labels Sep 23, 2026
x-at-01 added a commit to webc-site/wedb that referenced this pull request Sep 23, 2026
wbase::future::blocking_wait 在运行时上下文内嵌套 block_on,重入
compio-executor 的 tick(外层迭代器惰性预取被内层轮转失效),debug 构建炸
next_hot 的 item.is_hot 断言、release 静默断链丢调度。本提交消除 acl 与
cluster/replication/lua 全部同步收割点:

- acl 链:存储访问/命令处理/认证路径全 async 化,预门三态化(Permitted/
  Denied/Parked),AUTH/HELLO/ACL 走停车臂由网络泵 await 驱动
- cluster:publish 转 spawn detach(C# PubSubCommands.cs:148 同形态),
  checkpoint/sync 慢路径转 pending_slow 挂起(同文件五处先例同构)
- lua:VM 绑定(Luau C API lua_pcall)确属同步边界,C# 于回调栈上
  BlockingWait 内联收割(LuaRunner.Functions.cs:3239),新增
  wbase::future::inline_wait 对位——compio 低层 run/poll 驱动,依赖
  tick 重入安全(vendor compio-executor patch,上游
  compio-rs/compio#1058 合并后撤销)
- diskann 依赖切 webc-fork 发布版(webc-diskann* 0.59.0-webc.3,
  runtime 无关化:tokio/compio 双可选后端,compio 形态 tokio 出零次)

验收:wnode 1547 全绿、wedb cluster/replication/lua 216 全绿、clippy 零告警
x-at-01 added a commit to webc-site/wedb that referenced this pull request Sep 23, 2026
… patch 承接

- cleanup/quantization:专用会话守卫与共享索引锁随 async 栈帧覆盖,await 化闭环
  (panic 隔离改 FutureExt catch_unwind,对标 drive.rs 泵先例)
- 三处真同步调用根存留(如实登记于 wbase::future 文档):registry persistence
  (dyn 擦除+独占锁同步段,装箱 async 即同线程死锁)、StoreCallbacks 冷区读链
  (根在外部 webc-diskann DataProvider 同步契约,上游 inplace_delete async 块内
  同步调用,跨仓根治另立任务)
- 重入安全承接形态改 git patch:[patch.crates-io] compio-executor 指
  x-at-01/compio-fork fix/executor-tick-reentrant(=compio-rs/compio#1058 同源
  修复),vendor 目录不再存在;上游合并发版后删段回官方
- 验收:全量 4005/4005 全绿,clippy 零警告
The hot queue was walked with an iterator that lazily prefetches the
successor through `next_hot` while `tick`'s loop body had already moved
the previously returned task off the hot list (`make_cold` also clears
its `next`). Whenever a task's poll re-entered `tick` on the same
executor — nested driving, e.g. a synchronous harvest inside a
thread-per-core runtime — the inner tick could transition the prefetched
task from hot to cold. The outer iteration then either tripped
`debug_assert!(item.is_hot)` on debug builds or silently truncated the
walk on release builds, leaving queued tasks unpolled until the next
tick.

Handle the stale cursor explicitly: when the cursor's task is no longer
hot, restart from the current hot head; and in the loop body, skip tasks
that a nested tick has already taken off the hot list instead of
asserting or polling them a second time. Re-entry is now crash-free and
lossless, and the iteration still visits every queued task exactly once
per tick when no nesting occurs.
@x-at-01
x-at-01 force-pushed the fix/executor-tick-reentrant branch from d9721c2 to 916339b Compare September 24, 2026 03:59
@x-at-01

x-at-01 commented Sep 24, 2026

Copy link
Copy Markdown
Author

Closed in favor of re-forking under webc-fork

@x-at-01 x-at-01 closed this Sep 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working package: executor Related to compio-executor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants