GPU memory consumption

[wangyxxjtu]

Interesting work, how much GPU memory is required for fine-tuning the model?

[Lqf-HFNJU]

In our experimental setup, training on 8 A100 GPUs requires approximately 50GB of memory per GPU.

[wangyxxjtu]

In our experimental setup, training on 8 A100 GPUs requires approximately 50GB of memory per GPU.

thanks for your reply.