PyOpenCL version mismatch and kernel issues on different machines

[DarvinShehu]

Hi,
I’d like to ask for some guidance regarding an issue we’re facing with PyOpenCL when using the mdt Python package for brain imaging processing.

The package relies on PyOpenCL for computation. However, we’re observing inconsistent behavior when running the same script on two different machines, even though we’ve tried to align the software environments.

Here are the two setups:

Machine A (working, but slow):

OS: Fedora 39

CPU: Intel Xeon(R) E5-2650

GPU: NVIDIA Quadro P5000

Python: 3.7.1

OpenCL: 1.2

PyOpenCL: 2022.2.4

Machine B (hanging during execution):

OS: Oracle Linux 9.5

CPU: Intel Xeon(R) Silver 4116

GPU: NVIDIA GeForce RTX 3090

Python: 3.7.1

OpenCL: 1.2

PyOpenCL: 2022.2.4

On Machine A, everything runs correctly, though slowly—likely due to the older GPU. On Machine B, the process hangs at certain points.

Our current hypothesis is that the problem may be related to the older PyOpenCL version, which might not be fully compatible or optimized for newer hardware (such as the RTX 3090). We’re wondering:

Could this be caused by an incompatibility between the old PyOpenCL version and the newer GPU/kernels?

We’d really appreciate any insight or suggestions about what might be causing this issue.

Thank you in advance for your support!

[inducer]

A good first stop for debugging hangs of any kind is to look at the output of strace during/just before the apparent hang. This tells you what syscalls are happening, and, potentially, what's being waited on.

Another relevant bit of information would be which ICD you're using. From your description, it sounds like you might be using Nvidia's ICD? If so, could you report the version of that/the GPU driver?