Currently, easyconfigs of NVHPC only provide the compilers from Nvidia (nvc, nvc++, nvfortran, ...) and we build toolchains on top of those compilers with "standard" components. For instance, nvompi includes OpenMPI and nvofbf includes math libraries with FlexiBLAS.

However, the NVIDIA HPC SDK already provides its own components needed for a toolchain and the compilers from Nvidia expect those components to be available. This is the case for several -M compiler options, which are specifically designed for the NVHPC ecosystem.

For instance, the option -Mscalapack is defined in compilers/bin/rcfiles/lin86rc and translates to:

-lscalapack_ilp64 -llapack_ilp64 -lblas_ilp64 -lscalapack_lp64 -llapack_lp64 -lblas_lp64

Those libraries, although they seem to be numeric libraries, are only available in the OpenMPI distributed with NVHPC:

comm_libs/12.6/hpcx/hpcx-2.20/ompi/lib/libscalapack_ilp64.a
comm_libs/12.6/hpcx/hpcx-2.20/ompi/lib/libscalapack_ilp64.so.2.2.0
comm_libs/12.6/hpcx/hpcx-2.20/ompi/lib/libscalapack_ilp64.so.2
comm_libs/12.6/hpcx/hpcx-2.20/ompi/lib/libscalapack_ilp64.so
comm_libs/12.6/openmpi4/openmpi-4.1.5/lib/libscalapack_ilp64.so.2.2.0
comm_libs/12.6/openmpi4/openmpi-4.1.5/lib/libscalapack_ilp64.so.2
comm_libs/12.6/openmpi4/openmpi-4.1.5/lib/libscalapack_ilp64.so
comm_libs/12.6/openmpi4/openmpi-4.1.5/lib/libscalapack_ilp64.a
comm_libs/11.8/openmpi4/openmpi-4.1.5/lib/libscalapack_ilp64.so.2.2.0
comm_libs/11.8/openmpi4/openmpi-4.1.5/lib/libscalapack_ilp64.so
comm_libs/11.8/hpcx/hpcx-2.14/ompi/lib/libscalapack_ilp64.a
comm_libs/11.8/openmpi4/openmpi-4.1.5/lib/libscalapack_ilp64.so.2
comm_libs/11.8/openmpi4/openmpi-4.1.5/lib/libscalapack_ilp64.a
comm_libs/11.8/hpcx/hpcx-2.14/ompi/lib/libscalapack_ilp64.so.2.2.0
comm_libs/11.8/hpcx/hpcx-2.14/ompi/lib/libscalapack_ilp64.so.2
comm_libs/11.8/hpcx/hpcx-2.14/ompi/lib/libscalapack_ilp64.so
comm_libs/openmpi/openmpi-3.1.5/lib/libscalapack_ilp64.so.2.2.0
comm_libs/openmpi/openmpi-3.1.5/lib/libscalapack_ilp64.so.2
comm_libs/openmpi/openmpi-3.1.5/lib/libscalapack_ilp64.so
comm_libs/openmpi/openmpi-3.1.5/lib/libscalapack_ilp64.a

AFAIK these libraries are not available anywhere else, which means that building our own OpenMPI and math libraries in a toolchain like nvofbf is set for failure with software configured for NVHPC using those -M options.

Therefore, it will be more reliable to make NVHPC a complete toolchain using the components provided by Nvidia:

• current NVHPC easyconfigs should be renamed to something else, nvidia-compilers seems a good choice as we already have intel-compilers
• do we use NVHPC as a regular dependency using SYSTEM as toolchain, or do we make NVHPC an actual Toolchain in easybuild?
• templates for CUDA compute capabilities should be enabled on NVHPC as well