Sessions in MPI

[spophale]

Brief Description

MPI Sessions addresses a number of the limitations of the current MPI programming model. Among the immediate problems MPI Sessions is intended to address are the following:

• MPI cannot be initialized within an MPI process from different application components without a priori knowledge or coordination,
• MPI cannot be initialized more than once, and MPI cannot be reinitialized after MPI finalize has been called.

With MPI Sessions, an application no longer needs to explicitly call MPI_Init to make use of MPI, but rather can use a Session to only initialize MPI resources for specific communication needs. Unless the MPI process explicitly calls MPI_Init, there is also no explicit MPI_COMM_WORLD communicator. Sessions can be created and destroyed multiple times in an MPI process.

Use Case Details

Steps to create MPI Communicator from a Session Handle are shown below:

A PMIx Process Set is a user-provided or host environment assigned
label associated with a given set of application processes. Processes can
belong to multiple process sets at a time. Definition of a PMIx
process set typically occurs at time of application execution - e.g., on a
command line:
prun -n 4 --pset ocean myoceanapp : -n 3 --pset ice myiceapp

PMIx PSETs are used for query functions (MPI_SESSION_GET_NUM_PSETS,
MPI_SESSION_GET_NTH_PSET) and to create MPI_GROUP from a process set name.

Interfaces

PMIx_Get

Attributes/Directives

PMIX_PSET_NAMES    /* Returns an array of _char*_ string names of the process sets in which the given process is a member. */

PMIx groups are used during creation of MPI_COMM from an MPI_GROUP. The PMIx group constructor returns a 64-bit PMIx Group Context Identifier (PGCID) that is
guaranteed to be unique for the duration of an allocation (in the case of a batch managed
environment). This PGCID could be used as a direct replacement for the existing unique identifiers for communicators in MPI (E.g. Communicator Identifiers (CIDs) in Open MPI), but may have performance implications.

There is an important distinction between process sets and process groups. The process set identifiers are set by the host environment and currently there are no PMIx APIs provided by which an application can change a process set membership. In contrast, PMIx process groups can only be defined dynamically by the application.

Interfaces

PMIx_Group_construct

Return Values

- PMIX_SUCCESS           /* The request has been successfully completed */
- PMIX_ERR_NOT_SUPPORTED /* The PMIx library and/or the host RM does not support this operation. */

Attributes/Directives

PMIX_QUERY_NUM_GROUPS        /* Returns the number of process groups defined in the specified range (defaults to session). */
PMIX_QUERY_GROUP_NAMES       /* Returns a _pmix_data_array_t_ containing an array of string names of the process groups defined in the specified range (defaults to session). */
PMIX_QUERY_GROUP_MEMBERSHIP. /* Returns a _pmix_data_array_t_ of _pmix_proc_t_ containing the members of the specified process group.

References

• PMIx PR Modify Process Set Definition (Modify process set definitions #258)

• MPI Sessions: Second Demonstration and Evaluation of MPI Sessions Prototype (STPM13/OMPIX/13-35)

[hppritcha]

Might want to explain that CID stands for communication identifier used in Open MPI.

[jjhursey]

The v5.0.x PR #349 included this use case. The issue will remain open for further discussion on this topic.