inference memory

[sunyclj]

in the paper, "Unlike MIMO frameworks, our buffers work in a pipeline that consumes frames one by one. Therefore, our method can keep a constant run-time memory, which depends only on the number of Bidirectional Buffer Blocks 𝑁."
but,when testing,for 20 input images, the memory takes about 10G;for 40 input images, the memory takes about 14G.
Please, What's the reason?