Internal error using shmem_reduce in example/oshmem_max_reduction.c

[smguzik]

Background information

What version of Open MPI are you using? (e.g., v4.1.6, v5.0.1, git branch name and hash, etc.)

v5.0.2

Describe how Open MPI was installed (e.g., from a source/distribution tarball, from a git clone, from an operating system distribution package, etc.)

From source tarball using:
Configure command line: '--build=x86_64-linux-gnu'
'--prefix=/usr/local/openmpi/5.0.2_gcc-12.2.0'
'--with-ucx' '--with-pmix=internal'
'--with-libevent=external' '--with-hwloc=external'
'--enable-mpi-fortran=all'
'--with-cuda=/usr/local/cuda'
'--with-cuda-libdir=/usr/lib/x86_64-linux-gnu'

Please describe the system on which you are running

• Operating system/version: Debian 12.4
• Computer hardware: x86_64
• Network type: Single node

---

Details of the problem

oshmem_max_reduction.c works as provided in the examples directory. However, using the more recent API, replacing

shmem_long_max_to_all(dst, src, N, 0, 0, num_pes, pWrk, pSync);

with

shmem_long_max_reduce(SHMEM_TEAM_WORLD, dst, src, N);

fails with the message

[shmem_reduce.c:473:pshmem_long_max_reduce] Internal error is appeared rc = -7

[wenduwan]

@janjust I see --with-ucx - guess you would be interested 😄

[wenduwan]

Added main label assuming oshmem is the same with v5.0.x

[roiedanino]

It seems that the new API is not implemented yet in UCX spml module (or anywhere else):

From ucx/spml.c:1850

/* This routine is not implemented */
int mca_spml_ucx_team_reduce(shmem_team_t team, void
        *dest, const void *source, size_t nreduce, int operation, int datatype)
{
    return OSHMEM_ERR_NOT_IMPLEMENTED;
}

@MamziB Any chance I'm missing something? or it's a known TBD?

[popina1994]

I am having the same issue.
Should I use the old OpenSHMEM API or there is a way to bypass this?

[MamziB]

@roiedanino yeah we will implement this in the future.
@popina1994 Should I use the old OpenSHMEM API or there is a way to bypass this? yes please go ahead and use the old openshmem for now. if I find a better workaround I will update here.

[gleon99]

@MamziB can reassign to yourself please?

[gleon99]

@MamziB ?

[MamziB]

@gleon99 Sure let me assign it to myself. Thanks for reminder.