Indexer fails on reorg

[GA006]

Multichain indexer fails on mainnet reorg:

[15:33:59.890] INFO (71): Blockchain reorg detected. Initiating indexer rollback.
    chainId: 1
    blockNumber: 23369398
    indexedBlockHash: "0xc7260abb2e2d83a2bcdfdb2c769cf8229c612d0f75a4a41d05f48fb810964b4e"
    receivedBlockHash: "0xd4c3b05ff10e715c716888d428d563ae71f1ce2d9976b29f15a81a22fd32e30b"
[15:33:59.897] INFO (71): Invalidated action discarded
    action: "ValidatePartitionQueryResponse"
[15:33:59.927] INFO (71): Started rollback on reorg
    chainId: 1
    action: "Rollback"
    reorgChain: 1
    targetBlockNumber: 23369397
    targetBlockTimestamp: 1757950391
[15:33:59.965] INFO (71): Invalidated action discarded
    action: "FinishWaitingForNewBlock"
[15:34:01.025] INFO (71): Invalidated action discarded
    action: "FinishWaitingForNewBlock"
[15:34:01.144] INFO (71): Invalidated action discarded
    action: "FinishWaitingForNewBlock"
[15:34:02.653] INFO (71): Invalidated action discarded
    action: "ValidatePartitionQueryResponse"
[15:34:03.417] ERROR (71): Failed during batch delete of entity EUL_Transfer
    err: {
      "type": "PostgresError",
      "message": "current transaction is aborted, commands ignored until end of transaction block",
      "stack":
          PostgresError: current transaction is aborted, commands ignored until end of transaction block
              at ErrorResponse (/envio-indexer/generated/node_modules/.pnpm/postgres@3.4.1/node_modules/postgres/cjs/src/connection.js:783:26)
              at handle (/envio-indexer/generated/node_modules/.pnpm/postgres@3.4.1/node_modules/postgres/cjs/src/connection.js:475:6)
              at TLSSocket.data (/envio-indexer/generated/node_modules/.pnpm/postgres@3.4.1/node_modules/postgres/cjs/src/connection.js:315:9)
              at TLSSocket.emit (node:events:507:28)
              at TLSSocket.emit (node:domain:489:12)
              at addChunk (node:internal/streams/readable:559:12)
              at readableAddChunkPushByteMode (node:internal/streams/readable:510:3)
              at TLSSocket.Readable.push (node:internal/streams/readable:390:5)
              at TLSWrap.onStreamRead (node:internal/stream_base_commons:189:23)
              at cachedError (/envio-indexer/generated/node_modules/.pnpm/postgres@3.4.1/node_modules/postgres/cjs/src/query.js:170:23)
              at new Query (/envio-indexer/generated/node_modules/.pnpm/postgres@3.4.1/node_modules/postgres/cjs/src/query.js:36:24)
              at sql (/envio-indexer/generated/node_modules/.pnpm/postgres@3.4.1/node_modules/postgres/cjs/src/index.js:112:11)
              at Object.module.exports.batchDeleteItemsInTable (/envio-indexer/generated/src/db/DbFunctionsImplementation.js:23:15)
      "name": "PostgresError",
      "severity_local": "ERROR",
      "severity": "ERROR",
      "code": "25P02",
      "file": "postgres.c",
      "line": "1514",
      "routine": "exec_parse_message"
    }

The indexer works when restarted, but from checkpoint 23369398, so there may be data corruption.

[DZakh]

Thanks, I managed to create a reproduction test, I'll keep you updated about a fix #745

[DZakh]

I'm still working on a fix, and I have already merged 3 preparation PRs. As a workaround, you can use 2.30.1 with unordered multichain mode set to false.

[DZakh]

Here's the 4th #757