GPU with 22 GB but still got OutOfMemory error

[shirubei]

Hello, thanks for open source .
Running inference.py on 20808ti with 22GB but still got OutOfMemory error.

torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 32.00 GiB. GPU 0 has a total capacity of 22.00 GiB of which 0 bytes is free. Of the allocated memory 38.97 GiB is allocated by PyTorch, and 3.20 GiB is reserved by PyTorch but unallocated.

So how much GPU memory is required for infernece ?
Any suggestions to reduce memory usage, can we use quantization .

Thank you.

[ltzheng]

We can run it on 4090 with 24GB memory. Quantization might help but we have not tested the performance.

[h-oneohone]

same, I run it with 4090 24G but still out of memory. Have you solved it yet?

[talhaanwarch]

I tried running the sample image and audio in asset folder on 3090 and i am able to run it

[Deng555]

支持双卡跑吗？