Error about inferencing infer_MGPU.py on RTX4090

[thaondc-mbzuai]

I did try to run infer_MGPU.py with offload_switch=true. However, it still ran out of memory for A100 40 GB. Could you please check the provided source again?

[Kemb0]

Same here on a 4090. It literally says to change that for a 4090 but fails. I have had previous models where you could just free up a few more mb and they'd run but if a 40gb card isn't working then something more is wrong here.