Conditional loading of CUDA libaries

[jkarthic]

Right now if we build whisper with CUDA enabled, it is impossible to run the same binary on machines without nvidia GPU for CPU-only execution. For applications shipping whisper to consumer devices, this is a hassle as the apps have to ship two versions of whisper binaries with CUDA and without CUDA. This is due to the fact that CUDA libraries are linked dynamically, instead of loading conditionally(using dlopen on Linux or LoadLibrary on Windows) based on the presence of Nvidia GPU.
Is it possible to move to conditional loading of CUDA libraries and using function pointers for invoking CUDA functions? This will make the compiled binaries portable across multiple hardwares.

[ggerganov]

We are aware of this limitation and will work towards supporting runtime CPU detection and dynamic loading of the backends. No ETA for now - it might take some time

[jkarthic]

Thanks for the quick and clear reply!