What happened
On 2026-09-18 ~10:07 UTC the DCA feed posted this for schedule 37356 (https://hydration-explorer.neckwork.net/dca/37356):
🐕5ue split over 3 swaps 44 550 000 000 DOT for 557 500 000 000 000 EURC ~ Ħ640,900,000,000,000
The schedule actually sold 3 × ~166.7 DOT for ~500.4 EURC. Expected line: 499.9 DOT for 500.4 EURC.
It is not a decimals bug — the buffer summed another schedule's trades
The bogus sums are reproduced exactly by two executions of schedule 37352 (a different user, GDOT → HEURC, both 18-decimal assets) added to one real 37356 trade, then formatted with DOT (10) / EURC (6) decimals:
|
37352 @ 14743659 |
37352 @ 14743695 |
sum |
shown (4 sig. digits) |
| amountIn |
222 823 666 666 666 666 666 |
222 629 811 277 020 083 196 |
4.45453e20 |
44 550 000 000 DOT ✔ |
| amountOut |
279 381 419 274 171 975 596 |
278 070 002 708 535 048 880 |
5.57451e20 |
557 500 000 000 000 EURC ✔ |
(37352's middle execution @ 14743677 is not included — any combination including it rounds differently.) The Ħ value is just the inflated EURC figure × EURC price.
So broadcastBuffer(37356) in src/handlers/dca.js aggregated three entries: one genuine 37356 router.Executed (which supplied who and the DOT/EURC assets) plus two router.Executed events belonging to 37352 from other blocks.
The code on main does not reproduce it from canonical chain data
- Replayed blocks 14743640–14743735 (all 5ue / 8op DCAs of that window) through the full handler set, both sequentially and with production-style concurrent
emitFromBlock (blocks fired without awaiting). Every line correct, including 37352's own (668.3 GDOT for 836.5 HEURC) and 37356's (499.9 DOT for 500.4 EURC).
- In each of 37356's blocks (14743687 / 14743705 / 14743723) there is exactly one
router.Executed and one DCA in the initialization phase, so the "nearest preceding Executed" lookup cannot pick a wrong trade there.
- Buffer entries are filtered by schedule id; there is no path in this code that merges ids.
Second anomaly: the preceding "correct" line is also wrong
The line just before it read 499.9 DOT for 502.3 EURC ~ Ħ578. Schedule 37353 (the only candidate) sums to 502 370 204 → formats as 502.4. No combination of three trades from 37353/37356 yields 502.3. So the deployed instance saw amounts the canonical chain does not have for at least two consecutive messages.
Hypotheses (need production-side data)
- RPC inconsistency / fork.
rpc.hydradx.cloud is load-balanced. If getBlockHash(N) and system.events.at(hash) hit different nodes, or the node was on a short fork, the bot processes blocks the canonical chain never had; skipped/divergent blocks leave stale buffer entries. Would explain both the 502.3 and the leak.
- Deployed code ≠
main. Docker latest was built 2026-08-01 from main. If the container wasn't recreated or runs a different tag, the DCA handler may differ.
To confirm
- Container image digest and logs for ~10:05–10:07 UTC 2026-09-18 — look for
failed to load / processing of the event failed.
- Did Discord get an 8op line for 37352 (GDOT → HEURC) around then, and what did 37354 / 37355 / 37357 say?
Cheap hardening regardless of root cause
In broadcastBuffer, refuse to sum executions whose router.Executed (assetIn, assetOut) differ from the first entry's; fall back to per-trade lines and log the mismatch. That alone would have blocked this message.
🤖 Generated with Claude Code
What happened
On 2026-09-18 ~10:07 UTC the DCA feed posted this for schedule 37356 (https://hydration-explorer.neckwork.net/dca/37356):
The schedule actually sold 3 × ~166.7 DOT for ~500.4 EURC. Expected line:
499.9 DOT for 500.4 EURC.It is not a decimals bug — the buffer summed another schedule's trades
The bogus sums are reproduced exactly by two executions of schedule 37352 (a different user, GDOT → HEURC, both 18-decimal assets) added to one real 37356 trade, then formatted with DOT (10) / EURC (6) decimals:
44 550 000 000 DOT✔557 500 000 000 000 EURC✔(37352's middle execution @ 14743677 is not included — any combination including it rounds differently.) The Ħ value is just the inflated EURC figure × EURC price.
So
broadcastBuffer(37356)insrc/handlers/dca.jsaggregated three entries: one genuine 37356router.Executed(which suppliedwhoand the DOT/EURC assets) plus tworouter.Executedevents belonging to 37352 from other blocks.The code on
maindoes not reproduce it from canonical chain dataemitFromBlock(blocks fired without awaiting). Every line correct, including 37352's own (668.3 GDOT for 836.5 HEURC) and 37356's (499.9 DOT for 500.4 EURC).router.Executedand one DCA in the initialization phase, so the "nearest precedingExecuted" lookup cannot pick a wrong trade there.Second anomaly: the preceding "correct" line is also wrong
The line just before it read
499.9 DOT for 502.3 EURC ~ Ħ578. Schedule 37353 (the only candidate) sums to 502 370 204 → formats as 502.4. No combination of three trades from 37353/37356 yields 502.3. So the deployed instance saw amounts the canonical chain does not have for at least two consecutive messages.Hypotheses (need production-side data)
rpc.hydradx.cloudis load-balanced. IfgetBlockHash(N)andsystem.events.at(hash)hit different nodes, or the node was on a short fork, the bot processes blocks the canonical chain never had; skipped/divergent blocks leave stale buffer entries. Would explain both the 502.3 and the leak.main. Dockerlatestwas built 2026-08-01 from main. If the container wasn't recreated or runs a different tag, the DCA handler may differ.To confirm
failed to load/processing of the event failed.Cheap hardening regardless of root cause
In
broadcastBuffer, refuse to sum executions whoserouter.Executed(assetIn, assetOut)differ from the first entry's; fall back to per-trade lines and log the mismatch. That alone would have blocked this message.🤖 Generated with Claude Code