IOBinding: When to use?

[skaldesh]

I was just learning about IOBinding the other day and I have a basic question about that.
As far as I understand it, when I want to execute my model on a GPU, we need to somehow get the image data to the GPU. Without IOBinding, the copy from CPU to GPU is automatically done in the Run() call. And afterwards, when we want to interpret the output data, we need to copy the output data back from the GPU to the CPU.

If the above is my use case (where every input is different and every output needs to be analyzed on the CPU afterwards), would an IOBinding be of any use?
I currently only see a potential benefit, if I either have my input data already on the GPU, or need to process the output data on the GPU. That way, I can avoid the copies that would normally happen.

[pranavsharma]

In your use case, there's not much benefit.

[skaldesh]

Thank you

[seungtaek94]

@pranavsharma

Can you explain which case has an advantage when using IOBinding?

[pranavsharma]

https://onnxruntime.ai/docs/performance/tune-performance.html#iobinding