Repository navigation
Commit bfc70ea
committed
feat(infinity-slack-bot): resume streaming threads across restarts
Restarting the bot used to orphan in-flight responses: the daemon kept
generating, nothing reconnected until the user's next message, and the
output produced meanwhile never reached Slack (and dying mid-stream left
Slack messages dangling in streaming state).
## Clean shutdown (SIGTERM / ctrl-c)
* The inbound task closes the Socket Mode WebSocket with a proper close
handshake (Slack sees the app offline immediately; envelopes already read
were ACKed and forwarded, unread ones stay unACKed and are redelivered
after the restart — a racing user message is delayed, never lost).
* The outbound task drains: flushes buffered text, completes open task
cards, finalizes every active stream with the disclaimer footer, and
clears thread statuses. Queued-but-unprocessed actions are deliberately
dropped — they were never counted as delivered, so the restart replays
them. The whole drain is bounded by a 10s timeout.
* The resume state is persisted to `~/.infinity/slack_resume.json`
(atomic temp-file rename): per-stream delivery cursors, channel mappings,
subagent thread routes, user-stop latches, and posted choice buttons. A
periodic 30s snapshot bounds the damage of a crash (documented: a crash
may re-post or drop a few seconds of output).
## Delivery cursors
* `runtime.stream_progress` tracks, per (root thread, subagent), the
cumulative text bytes and tool call/result cards *delivered to Slack*,
counted by the I/O layer at delivery time (text when accepted into the
flush buffer — flushed before shutdown completes — cards when posted).
Suppressed (user-stopped) output is not counted and stays suppressed
across restarts via the persisted latch.
* `StreamStop` gains an `end_of_response` flag (set only by the
ResponseDone-driven stop): a response-final stop drops the stream's
cursor — cursor presence is exactly "this stream needs resuming".
## Startup resume
* Each persisted stream gets its own daemon connection
(`ConnectSession.target_thread`; replay — and an already-running
subagent driver's live broadcast — are per thread, and a new root
subscriber is not seeded into pre-existing child drivers).
* The daemon's `Replay` is trimmed by the pure `resume::replay_delta`:
skip the cursor-covered prefix (slicing inside replayed whole-message
text chunks on char boundaries; clamped if history shrank via rewind or
compaction), emit the remainder through the existing rendering arms, and
append the synthetic `ResponseDone`s replay history never carries —
counted to restore the exact end-state the live sequence would have
produced (tool-executing: none; completion in flight: one to consume the
round-continues flag; idle: one to close, plus one if the delta re-armed
the flag). Replays for non-resumed streams are dropped (their content is
already on screen).
* Still-pending choices are re-posted only if their buttons weren't already
on screen (restored `choice_messages`); pre-restart buttons keep
working. Subagent output resumes into its existing dedicated thread
(anchor map seeded from the persisted routes), and replies inside
subagent threads keep routing to their subagent.
## Tests & docs
* 10 unit tests for the resume store and `replay_delta` (trimming,
mid-chunk slicing, multibyte boundaries, independent call/result
cursors, synthetic-done counting, clamping, snapshot/restore round-trip).
* 2 new sim tests: the ResponseDone stop is marked end-of-response, and the
two-dones-close-a-replayed-tool-round contract `replay_delta` relies on.
* "Restarts and Continuity" section in the Slack bot docs.1 parent 70474b2 commit bfc70ea
11 files changed
Lines changed: 1732 additions & 97 deletions
File tree
- crates
- infinity-slack-bot/src
- infinity-slack-dataflow
- src
- tests
- docs/docs/infinity-code
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
| 16 | + | |
16 | 17 | | |
17 | 18 | | |
18 | 19 | | |
| |||
98 | 99 | | |
99 | 100 | | |
100 | 101 | | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
101 | 134 | | |
102 | 135 | | |
103 | 136 | | |
104 | 137 | | |
105 | 138 | | |
106 | | - | |
107 | | - | |
108 | | - | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
109 | 142 | | |
110 | 143 | | |
111 | 144 | | |
112 | | - | |
| 145 | + | |
113 | 146 | | |
114 | 147 | | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
115 | 186 | | |
116 | 187 | | |
117 | 188 | | |
| |||
136 | 207 | | |
137 | 208 | | |
138 | 209 | | |
139 | | - | |
| 210 | + | |
| 211 | + | |
140 | 212 | | |
141 | | - | |
| 213 | + | |
142 | 214 | | |
143 | 215 | | |
144 | 216 | | |
145 | | - | |
146 | | - | |
147 | | - | |
148 | | - | |
149 | | - | |
150 | | - | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
151 | 222 | | |
152 | 223 | | |
153 | 224 | | |
154 | 225 | | |
155 | | - | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
156 | 311 | | |
157 | 312 | | |
158 | 313 | | |
| |||
0 commit comments