GPU insights to solve FEM systems

[rebpoli]

Hi all,

I am wondering whether you guys have tried to get some speedup in libmesh with GPUs or any other architecture.
Do you have any resource or previous experience to share?

Thank you,
Renato

[roystgnr]

The MOOSE folks tried many years ago, and ran into the problem that you practically need to get all your compute onto the GPU; otherwise the cost of constantly moving data from GPU to CPU and back kills your speedup.

I tried not quite as many years ago using VexCL, and libMesh on GPUs worked well enough for very simple problems, but I never managed to make the API clean enough for regular users or flexible enough for real applications.

[rebpoli]

Thak you, Roy. I guess this might be a general observation. Does it sound like a barrier for mature code to be ported so that GPUs potential might be underexplored? Renato Em seg., 2 de out. de 2023 às 09:46, roystgnr ***@***.***> escreveu:

…

The MOOSE folks tried many years ago, and ran into the problem that you practically need to get *all* your compute onto the GPU; otherwise the cost of constantly moving data from GPU to CPU and back kills your speedup. I tried not quite as many years ago using VexCL, and libMesh on GPUs worked well enough for very simple problems, but I never managed to make the API clean enough for regular users or flexible enough for real applications. — Reply to this email directly, view it on GitHub <#3653 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMOVER2CUNCXZQCV4EUGNCLX5LHV7AVCNFSM6AAAAAA5N2PSMCVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TCNRWGA2DC> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[roystgnr]

"We have a legacy code with a zillion features that would have to be rewritten piece by piece to work on GPUs" is a common complaint, exacerbated by the "moving data from GPU to CPU and back kills your speedup" problem making it unprofitable to rewrite things one piece at a time. Also, nobody seems to like any of the abstraction layers for writing portable GPU-compatible code (I liked VexCL, but even my toy-problem-level tests required additions to that API, and I don't think it ever caught on in general), so the most popular thing to do now is still "write CUDA, then rewrite your backend in HIP too if you don't want to be locked in to NVidia".

I think https://developer.nvidia.com/blog/accelerating-standard-c-with-gpus-using-stdpar/ is moving in a promising direction, but the last time I played with it their compiler had a bug and it wouldn't quite pass all the libMesh test suite, which has me too scared to rely on it for applications.

[rebpoli]

Clear. Thank you. Em qua., 4 de out. de 2023 13:52, roystgnr ***@***.***> escreveu:

…

"We have a legacy code with a zillion features that would have to be rewritten piece by piece to work on GPUs" is a common complaint, exacerbated by the "moving data from GPU to CPU and back kills your speedup" problem making it unprofitable to rewrite things one piece at a time. Also, nobody seems to like any of the abstraction layers for writing portable GPU-compatible code (I liked VexCL, but even my toy-problem-level tests required additions to that API, and I don't think it ever caught on in general), so the most popular thing to do now is still "write CUDA, then rewrite your backend in HIP too if you don't want to be locked in to NVidia". I think https://developer.nvidia.com/blog/accelerating-standard-c-with-gpus-using-stdpar/ is moving in a promising direction, but the last time I played with it their compiler had a bug and it wouldn't quite pass all the libMesh test suite, which has me too scared to rely on it for applications. — Reply to this email directly, view it on GitHub <#3653 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMOVER3G276JIWCQXNEC4GLX5WV5RAVCNFSM6AAAAAA5N2PSMCVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TCOJQGQ2DM> . You are receiving this because you authored the thread.Message ID: ***@***.***>