NCCL tests hang when run in slurm interactive job

[abergeron]

This is due to those tests using mpiexec which attempts to start subprocesses with srun (if built with slurm support).

This hangs because srun says that job launching is "temporarily disabled" forever when in an interactive job. This could be fixed by forcing a different mpi launcher for the tests if we think it matters enough.

[nouiz]

Do you know how we can specify a different launcher? Is it easy?

[abergeron]

I remember that there is a simple parameter like --launcher but it's not that.

I don't remember what it was exactly and the mpi documentation/google/man pages are of course not at all helpful about this.