extending to non-NVIDIA GPU frameworks

[ahbarnett]

This is a place to discuss future plans for AMD, etc. GPUs.
We get questions more frequently about this, as cufinufft gets more supercomputer users.
Such as: "I am wondering if you and your team have any plans of extending the library to other GPU architectures such as AMD and Intel."

One ingredient would be switching out cuFFT for
https://github.com/icl-utk-edu/heffte
for example.

Another might be kokkos or openacc for the main code.

[DiamonDinoia]

To my knowledge, the performance of portability layers can vary significantly,for example in the context of High-Energy physics:1,2.

heffte appears to be a promising library, and it also offers a C interface, which should be relatively straightforward to integrate. Before implementation, I would benchmark it to ensure that its performance aligns with our expectations.

The primary motivation for leveraging portability layers is to circumvent the need for extensive code rewrites across multiple platforms. It's important to assess the extent of code replication that maintaining multiple platform-specific versions would require. Additionally, we could consider the option of re-implementing performance-critical code using native APIs while utilizing a portability layer for the remainder.

Another hypotesis to consider is if we can use FINUFFT load balanced spreader recusrively to minimize data transfers:

1. Spread across multiple nodes;
2. Spread across multiple GPUs;
3. Spread across multiple GPU-kernels.

Or in CPU only nodes:

1. Spread across multiple nodes;
2. Spread across multiple threads.

[blackwer]

Another hypotesis to consider is if we can use FINUFFT load balanced spreader recusrively to minimize data transfers:

I'd be happy to think about scaling this up -- but is anyone interested in doing HUGE nuffts? These things are a lot of development effort and won't pay off until you hit pretty extreme scales I don't think.

[blackwer]

Such as: "I am wondering if you and your team have any plans of extending the library to other GPU architectures such as AMD and Intel."

This has been on my radar for a while. It's actually straightforward and not much effort to get this working on an AMD GPU via HIP, but...

I'll be investigating SYCL for the GPU side of things since I want to learn it and it seems like a better long-term solution. I've seen quite a few reports that the performance is the same as native cuda, with free AMD/Intel support. If I can find a clean solution to resolve the whole "fft library" issue (cufft, rocfft, ???), then I'll probably make a test version to learn/benchmark.

edit1: it might turn out that heffte is a reasonable solution for all the FFT business, so I'm not forgetting that
edit2: We can get AMD support almost trivially since it is mostly cuda api compatible. There was even a PR for the old cufinufft codebase that did this...

[blackwer]

It isn't trivial work so I understand any reticence to fork. I just want to clarify a bit what the work would entail.

heFFTe and hipify are not mutually exclusive here. The bulk of cufinufft code is in the spreaders, which are implemented in cuda currently. Swapping out the fft implementation to heFFTe should be straightforward, but the rest of the code needs some kind of translation (syclomatic, hipify, re-write, etc) to drive it on other platforms. Our code is not using any real advanced cuda API calls, so the hipifystep should mostly just be getting it all to compile and possibly tuning it without (hopefully) much code writing. Swapping out for heFFTe would be replacing the various cufft_ex calls and initializing any structures related to that correctly if applicable. In most workflows we've seen that the fft call is a nearly negligible fraction of the actual work though.

For reference: there is an example PR for HIP that we unfortunately never merged before the great finufft-cufinufft merge and refactor https://github.com/flatironinstitute/cufinufft/pull/116/files . We use cmake now, but the gist is the same.

[srikrrish]

I see.. I now understand better. So what you are basically saying is since the complexity of NUFFT is O(\epsilon^d N_p + N_mlogN_m), where N_p is the no. of particles and N_m is the no. of Fourier modes. In most of the cases, the N_p term dominates and even if we replace the fft calls with heffte we still need to do the spread and gather (I am using PIC terminology here forgot what you guys call this but I think you understand) with cuda API calls or equivalent API calls for AMD or Intel GPUs. Is my understanding correct?

[srikrrish]

If that is the case I agree with you that HIPIFY should mostly do the job for at least AMD GPUs..

[srikrrish]

One more thing I always wanted to ask is, the reason why you guys (or basically most of the people who do NUFFT) don't go for distributed FFTs is, in most cases the particle cost dominates and simply dividing the particles equally to no. of cores/GPUs and doing an all reduce at the end of the NUFFT type 1 is good enough? This is how I am basically using FINUFFT also. Because if we start doing domain decompositions on the grids used in the FFTs then the no. of halo layers to communicate is a lot (when \epsilon is small and since for your kernels the width goes like |log(\epsilon)| + 1) and it may not be worth while. But I also see that it should depend on the no. of particles to the no. of modes needed right? In cases where the local particle work is not sufficient enough I can see having a distributed FFT (to reduce the N_mlog(N_m) cost) might be helpful. Of course then it depends on the use case.

[blackwer]

That's essentially the idea. The spreader and interpolation routines aren't written in a language that compiles for AMD or Intel gpus, so that code needs to be translated to a language that can. HIP is the language for AMD gpus directly. HIP is also basically a large subset of cuda, with slightly different function names, so the translation is straightforward (and you can, with C-macro magic, switch between the cuda and HIP names of functions to compile for NVIDIA or AMD with the same code). AMD provides HIP versions of cufft, so that base is automatically covered, though we can still swap it out as well.

Re: Intel: There also is some tooling to compile HIP -> Intermediate Language (SPIR-V) -> Intel compatible binary as I currently understand it. I haven't personally tried this though on a simple test code -- it's on my agenda. I'm not aware of a cufft-style compatible library for intel, so this is where i think heffte would be absolutely necessary.

See: https://clang.llvm.org/docs/HIPSupport.html for more details about the current state of things