Chap_API_Publish.tex

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Chapter: Publish/Lookup Operations
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\chapter{Publish/Lookup Operations}
\label{chap:pub}

Chapter~\ref{chap:api_rsvd_keys} and Chapter~\ref{chap:nrkeys} discussed how reserved and non-reserved keys dealt with
information that either was associated with a specific process (i.e., the
retrieving process knew the identifier of the process that posted it) or
required a synchronization operation prior to retrieval (e.g., the case of
globally unique non-reserved keys). However, another requirement exists for an
asynchronous exchange of data where neither the posting nor the retrieving
process is known in advance. For example, two separate namespaces may need to
rendezvous with each other without knowing in advance the identity of the other
namespace or when that namespace might become active.

The \acp{API} defined in this section focus on resolving that specific
situation by allowing processes to publish data that can subsequently be
retrieved solely by referral to its key. Mechanisms for constraining
availability of the information are also provided as a means for better
targeting of the eventual recipient(s).

Note that no presumption is made regarding how the published information is to be stored, nor as to the entity (host environment or \ac{PMIx} implementation) that shall act as the datastore. The descriptions in the remainder of this chapter shall simply refer to that entity as the \emph{datastore}.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{\code{PMIx_Publish}}
\declareapi{PMIx_Publish}

%%%%
\summary

Publish data for later access via \refapi{PMIx_Lookup}.

%%%%
\format

\copySignature{PMIx_Publish}{1.0}{
pmix_status_t \\
PMIx_Publish(const pmix_info_t info[], size_t ninfo);
}

\begin{arglist}
\argin{info}{Array of info structures containing both data to be published and directives (array of handles)}
\argin{ninfo}{Number of elements in the \refarg{info} array (integer)}
\end{arglist}

Returns \refconst{PMIX_SUCCESS} or a negative value corresponding to a PMIx error constant.

\reqattrstart
There are no required attributes for this \ac{API}. \ac{PMIx} implementations that do not directly support the operation but are hosted by environments that do support it must pass any attributes that are provided by the client to the host environment for processing. In addition, the \ac{PMIx} library is required to add the \refAttributeItem{PMIX_USERID} and the \refAttributeItem{PMIX_GRPID} attributes of the client process that published the information to the \refarg{info} array passed to the host environment.

\reqattrend

\optattrstart
The following attributes are optional for host environments that support this operation:

\pasteAttributeItem{PMIX_TIMEOUT}
\pasteAttributeItem{PMIX_RANGE}
\pasteAttributeItem{PMIX_PERSISTENCE}
\pasteAttributeItem{PMIX_ACCESS_PERMISSIONS}

\optattrend

%%%%
\descr

Publish the data in the \refarg{info} array for subsequent lookup.
By default, the data will be published into the \refconst{PMIX_RANGE_SESSION} range and with \refconst{PMIX_PERSIST_APP} persistence.
Changes to those values, and any additional directives, can be included in the \refstruct{pmix_info_t} array. Attempts to access the data by processes outside of the provided data range shall be rejected. The \refattr{PMIX_PERSISTENCE} attribute instructs the datastore holding the published information as to how long that information is to be retained.

The blocking form of this call will block until it has obtained confirmation from the datastore that the data is available for lookup. The \refarg{info} array can be released upon return from the blocking function call.

Publishing duplicate keys is permitted provided they are published to different
ranges. Duplicate keys being published on the same data range shall return the
\refconst{PMIX_ERR_DUPLICATE_KEY} error.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{\code{PMIx_Publish_nb}}
\declareapi{PMIx_Publish_nb}

%%%%
\summary

Nonblocking \refapi{PMIx_Publish} routine.

%%%%
\format

\copySignature{PMIx_Publish_nb}{1.0}{
pmix_status_t \\
PMIx_Publish_nb(const pmix_info_t info[], size_t ninfo, \\
\hspace*{16\sigspace}pmix_op_cbfunc_t cbfunc, void *cbdata);
}

\begin{arglist}
\argin{info}{Array of info structures containing both data to be published and directives (array of handles)}
\argin{ninfo}{Number of elements in the \refarg{info} array (integer)}
\argin{cbfunc}{Callback function \refapi{pmix_op_cbfunc_t} (function reference)}
\argin{cbdata}{Data to be passed to the callback function (memory reference)}
\end{arglist}

Returns one of the following:

\begin{itemize}
    \item \refconst{PMIX_SUCCESS}, indicating that the request is being processed by the host environment - result will be returned in the provided \refarg{cbfunc}. Note that the library must not invoke the callback function prior to returning from the \ac{API}.
    \item \refconst{PMIX_OPERATION_SUCCEEDED}, indicating that the request was immediately processed and returned \textit{success} - the \refarg{cbfunc} will \textit{not} be called.
    \item a PMIx error constant indicating either an error in the input or that the request was immediately processed and failed - the \refarg{cbfunc} will \textit{not} be called.
\end{itemize}

\reqattrstart
There are no required attributes for this \ac{API}. \ac{PMIx} implementations that do not directly support the operation but are hosted by environments that do support it must pass any attributes that are provided by the client to the host environment for processing. In addition, the \ac{PMIx} library is required to add the \refAttributeItem{PMIX_USERID} and the \refAttributeItem{PMIX_GRPID} attributes of the client process that published the information to the \refarg{info} array passed to the host environment.

\reqattrend

\optattrstart
The following attributes are optional for host environments that support this operation:

\pasteAttributeItem{PMIX_TIMEOUT}
\pasteAttributeItem{PMIX_RANGE}
\pasteAttributeItem{PMIX_PERSISTENCE}
\pasteAttributeItem{PMIX_ACCESS_PERMISSIONS}

\optattrend

%%%%
\descr

Nonblocking \refapi{PMIx_Publish} routine.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Publish-specific constants}

The following constants are defined for use with the \refapi{PMIx_Publish} \acp{API}:

\begin{constantdesc}
%
\declareconstitemNEW{PMIX_ERR_DUPLICATE_KEY}
The provided key has already been published on the same data range.
%
\end{constantdesc}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Publish-specific attributes}

The following attributes are defined for use with the \refapi{PMIx_Publish} \acp{API}:

%
\declareAttribute{PMIX_RANGE}{"pmix.range"}{pmix_data_range_t}{
Define constraints on the processes that can access the provided data. Only processes that meet the constraints are allowed to access it.
}
%
\declareAttribute{PMIX_PERSISTENCE}{"pmix.persist"}{pmix_persistence_t}{
Declare how long the datastore shall retain the provided data. The datastore is to delete the data upon reaching the persistence criterion.
}
%
\declareAttributeNEW{PMIX_ACCESS_PERMISSIONS}{"pmix.aperms"}{pmix_data_array_t}{
Define access permissions for the published data. The value shall contain an array of \refstruct{pmix_info_t} structs containing the specified permissions.
}
%
\declareAttributeNEW{PMIX_ACCESS_USERIDS}{"pmix.auids"}{pmix_data_array_t}{
Array of effective \acp{UID} that are allowed to access the published data.
}
%
\declareAttributeNEW{PMIX_ACCESS_GRPIDS}{"pmix.agids"}{pmix_data_array_t}{
Array of effective \acp{GID} that are allowed to access the published data.
}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Publish-Lookup Datatypes}

The following data types are defined for use with the \refapi{PMIx_Publish} \acp{API}.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Range of Published Data}
\declarestruct{pmix_data_range_t}

\versionMarker{1.0}
The \refstruct{pmix_data_range_t} structure is a \code{uint8_t} type that defines a range for both data \textit{published} via the \refapi{PMIx_Publish} \ac{API} and generated events.
The following constants can be used to set a variable of the type \refstruct{pmix_data_range_t}.

\begin{constantdesc}
%
\declareconstitem{PMIX_RANGE_UNDEF}
Undefined range.
%
\declareconstitem{PMIX_RANGE_RM}
Data is intended for the host environment, or lookup is restricted to data published by the host environment.
%
\declareconstitem{PMIX_RANGE_LOCAL}
Data is only available to processes on the local node, or lookup is restricted to data published by processes on the local node of the requester.
%
\declareconstitem{PMIX_RANGE_NAMESPACE}
Data is only available to processes in the same namespace, or lookup is restricted to data published by processes in the same namespace as the requester.
%
\declareconstitem{PMIX_RANGE_SESSION}
Data is only available to all processes in the session, or lookup is restricted to data published by other processes in the same session as the requester.
%
\declareconstitem{PMIX_RANGE_GLOBAL}
Data is available to all processes, or lookup is open to data published by anyone.
%
\declareconstitem{PMIX_RANGE_CUSTOM}
Data is available only to processes as specified in the \refstruct{pmix_info_t} associated with this call, or lookup is restricted to data published by processes as specified in the \refstruct{pmix_info_t}.
%
\declareconstitem{PMIX_RANGE_PROC_LOCAL}
Data is only available to this process, or lookup is restricted to data published by this process.
%
\declareconstitem{PMIX_RANGE_INVALID}
Invalid value - typically used to indicate that a range has not yet been set.
%
\end{constantdesc}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Data Persistence Structure}
\declarestruct{pmix_persistence_t}

\versionMarker{1.0}
The \refstruct{pmix_persistence_t} structure is a \code{uint8_t} type that defines the policy for data published by clients via the \refapi{PMIx_Publish} \ac{API}.
The following constants can be used to set a variable of the type \refstruct{pmix_persistence_t}.

\begin{constantdesc}
%
\declareconstitem{PMIX_PERSIST_INDEF}
Retain data until specifically deleted.
%
\declareconstitem{PMIX_PERSIST_FIRST_READ}
Retain data until the first access, then the data is deleted.
%
\declareconstitem{PMIX_PERSIST_PROC}
Retain data until the publishing process terminates.
%
\declareconstitem{PMIX_PERSIST_APP}
Retain data until the application terminates.
%
\declareconstitem{PMIX_PERSIST_SESSION}
Retain data until the session/allocation terminates.
%
\declareconstitem{PMIX_PERSIST_INVALID}
Invalid value - typically used to indicate that a persistence has not yet been set.
%
\end{constantdesc}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{\code{PMIx_Lookup}}
\declareapi{PMIx_Lookup}

%%%%
\summary

Lookup information published by this or another process with \refapi{PMIx_Publish} or \refapi{PMIx_Publish_nb}.

%%%%
\format

\copySignature{PMIx_Lookup}{1.0}{
pmix_status_t \\
PMIx_Lookup(pmix_pdata_t data[], size_t ndata, \\
\hspace*{12\sigspace}const pmix_info_t info[], size_t ninfo);
}

\begin{arglist}
\arginout{data}{Array of publishable data structures (array of \refstruct{pmix_pdata_t})}
\argin{ndata}{Number of elements in the \refarg{data} array (integer)}
\argin{info}{Array of info structures (array of \refstruct{pmix_info_t})}
\argin{ninfo}{Number of elements in the \refarg{info} array (integer)}
\end{arglist}

Returns one of the following:

\begin{itemize}
\item \refconst{PMIX_SUCCESS} All data was found and has been returned.

\item \refconst{PMIX_ERR_NOT_FOUND} None of the requested data could be found within the requester's range.

\item \refconst{PMIX_ERR_PARTIAL_SUCCESS} Some of the requested data was found.
Any key that cannot be found will return with a data type of \refconst{PMIX_UNDEF} in the associated \refarg{value} struct. Note that the specific reason for a particular piece of missing information (e.g., lack of permissions) cannot be communicated back to the requester in this situation.

\item \refconst{PMIX_ERR_NOT_SUPPORTED} There is no available datastore (either at the host environment or \ac{PMIx} implementation level) on this system that supports this function.

\item \refconst{PMIX_ERR_NO_PERMISSIONS} All of the requested data was found and range restrictions were met for each specified key, but none of the matching data could be returned due to lack of access permissions.

\item a non-zero \ac{PMIx} error constant indicating a reason for the request's failure.
\end{itemize}

\reqattrstart
\ac{PMIx} libraries are not required to directly support any attributes for this function. However, any provided attributes must be passed to the host environment for processing, and the \ac{PMIx} library is required to add the \refAttributeItem{PMIX_USERID} and the \refAttributeItem{PMIX_GRPID} attributes of the client process that is requesting the info.

\reqattrend

\optattrstart
The following attributes are optional for host environments that support this operation:

\pasteAttributeItem{PMIX_TIMEOUT}
\pasteAttributeItem{PMIX_RANGE}
\pasteAttributeItem{PMIX_WAIT}

\optattrend

%%%%
\descr

Lookup information published by this or another process.
By default, the search will be constrained to publishers that fall within the \refconst{PMIX_RANGE_SESSION} range in case duplicate keys exist on different ranges.
Changes to the range (e.g., expanding the search to all potential publishers via the \refconst{PMIX_RANGE_GLOBAL} constant), and any additional directives, can be provided in the \refstruct{pmix_info_t} array. Data is returned per the retrieval rules of Section \ref{chap:pub:retrules}.

The \argref{data} parameter consists of an array of \refstruct{pmix_pdata_t} structures with the keys specifying the requested information.
Data will be returned for each \code{key} field in the associated \code{value} field of this structure as per the above description of return values. The \code{proc} field in each \refstruct{pmix_pdata_t} structure will contain the namespace/rank of the process that published the data.

\adviceuserstart
Although this is a blocking function, it will not wait by default for the requested data to be published.
Instead, it will block for the time required by the datastore to lookup its current data and return any found items.
Thus, the caller is responsible for either ensuring that data is published prior to executing a lookup, using \refattr{PMIX_WAIT} to instruct the datastore to wait for the data to be published, or retrying until the requested data is found.
\adviceuserend


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{\code{PMIx_Lookup_nb}}
\declareapi{PMIx_Lookup_nb}

%%%%
\summary

Nonblocking version of \refapi{PMIx_Lookup}.

%%%%
\format

\copySignature{PMIx_Lookup_nb}{1.0}{
pmix_status_t \\
PMIx_Lookup_nb(char **keys, \\
\hspace*{15\sigspace}const pmix_info_t info[], size_t ninfo, \\
\hspace*{15\sigspace}pmix_lookup_cbfunc_t cbfunc, void *cbdata);
}

\begin{arglist}
\argin{keys}{\code{NULL}-terminated array of keys (array of strings)}
\argin{info}{Array of info structures (array of handles)}
\argin{ninfo}{Number of elements in the \refarg{info} array (integer)}
\argin{cbfunc}{Callback function (handle)}
\argin{cbdata}{Callback data to be provided to the callback function (pointer)}
\end{arglist}

Returns one of the following:

\begin{itemize}
    \item \refconst{PMIX_SUCCESS}, indicating that the request is being processed by the host environment - result will be returned in the provided \refarg{cbfunc}. Note that the library must not invoke the callback function prior to returning from the \ac{API}.
    \item a PMIx error constant indicating an error in the input - the \refarg{cbfunc} will \textit{not} be called.
\end{itemize}

If executed, the status returned in the provided callback function will be one of the following constants:

\begin{itemize}
\item \refconst{PMIX_SUCCESS} All data was found and has been returned.

\item \refconst{PMIX_ERR_NOT_FOUND} None of the requested data was available within the requester's range. The \refarg{pdata} array in the callback function shall be \code{NULL} and the \refarg{npdata} parameter set to zero.

\item \refconst{PMIX_ERR_PARTIAL_SUCCESS} Some of the requested data was found.
Only found data will be included in the returned \refarg{pdata} array. Note that the specific reason for a particular piece of missing information (e.g., lack of permissions) cannot be communicated back to the requester in this situation.

\item \refconst{PMIX_ERR_NOT_SUPPORTED} There is no available datastore (either at the host environment or \ac{PMIx} implementation level) on this system that supports this function.

\item \refconst{PMIX_ERR_NO_PERMISSIONS} All of the requested data was found and range restrictions were met for each specified key, but none of the matching data could be returned due to lack of access permissions.

\item a non-zero \ac{PMIx} error constant indicating a reason for the request's failure.
\end{itemize}

\reqattrstart
\ac{PMIx} libraries are not required to directly support any attributes for this function. However, any provided attributes must be passed to the host environment for processing, and the \ac{PMIx} library is required to add the \refAttributeItem{PMIX_USERID} and the \refAttributeItem{PMIX_GRPID} attributes of the client process that is requesting the info.

\reqattrend

\optattrstart
The following attributes are optional for host environments that support this operation:

\pasteAttributeItem{PMIX_TIMEOUT}
\pasteAttributeItem{PMIX_RANGE}
\pasteAttributeItem{PMIX_WAIT}

\optattrend

%%%%
\descr

Non-blocking form of the \refapi{PMIx_Lookup} function.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Lookup Returned Data Structure}
\declarestruct{pmix_pdata_t}

The \refstruct{pmix_pdata_t} structure is used by \refapi{PMIx_Lookup} to describe the data being accessed.

\copySignature{pmix_pdata_t}{1.0}{
typedef struct pmix_pdata \{ \\
\hspace*{4\sigspace}pmix_proc_t proc; \\
\hspace*{4\sigspace}pmix_key_t key; \\
\hspace*{4\sigspace}pmix_value_t value; \\
\} pmix_pdata_t;
}

where:
\begin{itemize}
    \item \emph{proc} is the process identifier of the data publisher.
    \item \emph{key} is the string key of the published data.
    \item \emph{value} is the value associated with the \emph{key}.
\end{itemize}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsubsection{Lookup data structure support macros}

The following macros are provided to support the \refstruct{pmix_pdata_t} structure.

\littleheader{Initialize the pdata structure}
\declaremacro{PMIX_PDATA_CONSTRUCT}

Initialize the \refstruct{pmix_pdata_t} fields

\copySignature{PMIX_PDATA_CONSTRUCT}{1.0}{
PMIX_PDATA_CONSTRUCT(m)
}

\begin{arglist}
\argin{m}{Pointer to the structure to be initialized (pointer to \refstruct{pmix_pdata_t})}
\end{arglist}

\littleheader{Destruct the pdata structure}
\declaremacro{PMIX_PDATA_DESTRUCT}

Destruct the \refstruct{pmix_pdata_t} fields

\copySignature{PMIX_PDATA_DESTRUCT}{1.0}{
PMIX_PDATA_DESTRUCT(m)
}

\begin{arglist}
\argin{m}{Pointer to the structure to be destructed (pointer to \refstruct{pmix_pdata_t})}
\end{arglist}

%%%%%%%%%%%
\littleheader{Create a pdata array}
\declaremacro{PMIX_PDATA_CREATE}

Allocate and initialize an array of \refstruct{pmix_pdata_t} structures

\copySignature{PMIX_PDATA_CREATE}{1.0}{
PMIX_PDATA_CREATE(m, n)
}

\begin{arglist}
\arginout{m}{Address where the pointer to the array of \refstruct{pmix_pdata_t} structures shall be stored (handle)}
\argin{n}{Number of structures to be allocated (\code{size_t})}
\end{arglist}


%%%%%%%%%%%
\littleheader{Free a pdata structure}
\declaremacro{PMIX_PDATA_RELEASE}

Release a \refstruct{pmix_pdata_t} structure

\copySignature{PMIX_PDATA_RELEASE}{4.0}{
PMIX_PDATA_RELEASE(m)
}

\begin{arglist}
\argin{m}{Pointer to a \refstruct{pmix_pdata_t} structure (handle)}
\end{arglist}


%%%%%%%%%%%
\littleheader{Free a pdata array}
\declaremacro{PMIX_PDATA_FREE}

Release an array of \refstruct{pmix_pdata_t} structures

\copySignature{PMIX_PDATA_FREE}{1.0}{
PMIX_PDATA_FREE(m, n)
}

\begin{arglist}
\argin{m}{Pointer to the array of \refstruct{pmix_pdata_t} structures (handle)}
\argin{n}{Number of structures in the array (\code{size_t})}
\end{arglist}

%%%%%%%%%%%
\littleheader{Load a lookup data structure}
\declaremacro{PMIX_PDATA_LOAD}

This macro simplifies the loading of key, process identifier, and data into a \refstruct{pmix_pdata_t} by correctly assigning values to the structure's fields.

\copySignature{PMIX_PDATA_LOAD}{1.0}{
PMIX_PDATA_LOAD(m, p, k, d, t);
}

\begin{arglist}
\argin{m}{Pointer to the \refstruct{pmix_pdata_t} structure into which the key and data are to be loaded (pointer to \refstruct{pmix_pdata_t})}
\argin{p}{Pointer to the \refstruct{pmix_proc_t} structure containing the identifier of the process being referenced (pointer to \refstruct{pmix_proc_t})}
\argin{k}{String key to be loaded - must be less than or equal to \refconst{PMIX_MAX_KEYLEN} in length (handle)}
\argin{d}{Pointer to the data value to be loaded (handle)}
\argin{t}{Type of the provided data value (\refstruct{pmix_data_type_t})}
\end{arglist}

\adviceuserstart
Key, process identifier, and data will all be copied into the \refstruct{pmix_pdata_t} - thus, the source information can be modified or free'd without affecting the copied data once the macro has completed.
\adviceuserend

%%%%%%%%%%%
\littleheader{Transfer a lookup data structure}
\declaremacro{PMIX_PDATA_XFER}

This macro simplifies the transfer of key, process identifier, and data value between two\refstruct{pmix_pdata_t} structures.

\copySignature{PMIX_PDATA_XFER}{2.0}{
PMIX_PDATA_XFER(d, s);
}

\begin{arglist}
\argin{d}{Pointer to the destination \refstruct{pmix_pdata_t} (pointer to \refstruct{pmix_pdata_t})}
\argin{s}{Pointer to the source \refstruct{pmix_pdata_t} (pointer to \refstruct{pmix_pdata_t})}
\end{arglist}

\adviceuserstart
Key, process identifier, and data will all be copied into the destination \refstruct{pmix_pdata_t} - thus, the source \refstruct{pmix_pdata_t} may free'd without affecting the copied data once the macro has completed.
\adviceuserend


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Lookup Callback Function}
\declareapi{pmix_lookup_cbfunc_t}

%%%%
\summary

The \refapi{pmix_lookup_cbfunc_t} is used by \refapi{PMIx_Lookup_nb} to return data.

\copySignature{pmix_lookup_cbfunc_t}{1.0}{
typedef void (*pmix_lookup_cbfunc_t) \\
\hspace*{4\sigspace}(pmix_status_t status, \\
\hspace*{5\sigspace}pmix_pdata_t data[], size_t ndata, \\
\hspace*{5\sigspace}void *cbdata);
}

\begin{arglist}
\argin{status}{Status associated with the operation (handle)}
\argin{data}{Array of data returned (\refstruct{pmix_pdata_t})}
\argin{ndata}{Number of elements in the \argref{data} array (\code{size_t})}
\argin{cbdata}{Callback data passed to original API call (memory reference)}
\end{arglist}


%%%%
\descr

A callback function for calls to \refapi{PMIx_Lookup_nb}.
The function will be called upon completion of the \refapi{PMIx_Lookup_nb} \ac{API} with the \refarg{status} indicating the success or failure of the request.
Any retrieved data will be returned in an array of \refstruct{pmix_pdata_t} structs.
The namespace and rank of the process that provided each data element is also returned.

Note that the \refstruct{pmix_pdata_t} structures will be released upon return from the callback function, so the receiver must copy/protect the data prior to returning if it needs to be retained.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Retrieval rules for published data}
\label{chap:pub:retrules}

The retrieval rules for published data primarily revolve around enforcing data access permissions and range constraints. The datastore shall search its stored information for each specified key according to the following precedence logic:

\begin{enumerate}
    \item If the requester specified the range, then the search shall be constrained to data where the publishing process falls within the specified range.

    \item If the key of the stored information does not match the specified key, then the search will continue.

    \item If the requester's identifier does not fall within the range specified by the publisher, then the search will continue.

    \item If the publisher specified access permissions, the effective \ac{UID} and \ac{GID} of the requester shall be checked against those permissions, with the datastore rejecting the match if the requester fails to meet the requirements.

    \item If all of the above checks pass, then the value is added to the information that is to be returned.
\end{enumerate}

The status returned by the datastore shall be set to:

\begin{itemize}
\item \refconst{PMIX_SUCCESS} All data was found and is included in the returned information.

\item \refconst{PMIX_ERR_NOT_FOUND} None of the requested data could be found within a requester's range.

\item \refconst{PMIX_ERR_PARTIAL_SUCCESS} Some of the requested data was found.
Only found data will be included in the returned information. Note that the specific reason for a particular piece of missing information (e.g., lack of permissions) cannot be communicated back to the requester in this situation.

\item a non-zero \ac{PMIx} error constant indicating a reason for the request's failure.
\end{itemize}

In the case where data was found and range restrictions were met for each specified key, but none of the matching data could be returned due to lack of access permissions, the datastore must return the  \refconst{PMIX_ERR_NO_PERMISSIONS} error.

\adviceuserstart
Note that duplicate keys are allowed to exist on different ranges, and that ranges do overlap each other. Thus, if duplicate keys are published on overlapping ranges, it is possible for the datastore to successfully find multiple responses for a given key should publisher and requester specify sufficiently broad ranges. In this situation, the choice of resolving the duplication is left to the datastore implementation - e.g., it may return the first value found in its search, or the value corresponding to the most limited range of the found values, or it may choose to simply return an error.

Users are advised to avoid this ambiguity by careful selection of key values and ranges - e.g., by creating range-specific keys where necessary.
\adviceuserend


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{\code{PMIx_Unpublish}}
\declareapi{PMIx_Unpublish}

%%%%
\summary

Unpublish data posted by this process using the given keys.

%%%%
\format

\copySignature{PMIx_Unpublish}{1.0}{
pmix_status_t \\
PMIx_Unpublish(char **keys, \\
\hspace*{15\sigspace}const pmix_info_t info[], size_t ninfo);
}

\begin{arglist}
\argin{keys}{\code{NULL}-terminated array of keys (array of strings)}
\argin{info}{Array of info structures (array of handles)}
\argin{ninfo}{Number of elements in the \refarg{info} array (integer)}
\end{arglist}

Returns \refconst{PMIX_SUCCESS} or a negative value corresponding to a PMIx error constant.

\reqattrstart
\ac{PMIx} libraries are not required to directly support any attributes for this function. However, any provided attributes must be passed to the host environment for processing, and the \ac{PMIx} library is required to add the \refAttributeItem{PMIX_USERID} and the \refAttributeItem{PMIX_GRPID} attributes of the client process that is requesting the operation.

\reqattrend

\optattrstart
The following attributes are optional for host environments that support this operation:

\pasteAttributeItem{PMIX_TIMEOUT}
\pasteAttributeItem{PMIX_RANGE}

\optattrend

%%%%
\descr

Unpublish data posted by this process using the given \refarg{keys}.
The function will block until the data has been removed by the server (i.e., it is safe to publish that key again within the specified range).
A value of \code{NULL} for the \refarg{keys} parameter instructs the server to remove all data published by this process.

By default, the range is assumed to be \refconst{PMIX_RANGE_SESSION}.
Changes to the range, and any additional directives, can be provided in the \refarg{info} array.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{\code{PMIx_Unpublish_nb}}
\declareapi{PMIx_Unpublish_nb}

%%%%
\summary

Nonblocking version of \refapi{PMIx_Unpublish}.

%%%%
\format

\copySignature{PMIx_Unpublish_nb}{1.0}{
pmix_status_t \\
PMIx_Unpublish_nb(char **keys, \\
\hspace*{18\sigspace}const pmix_info_t info[], size_t ninfo, \\
\hspace*{18\sigspace}pmix_op_cbfunc_t cbfunc, void *cbdata);
}

\begin{arglist}
\argin{keys}{\code{NULL}-terminated array of keys (array of strings)}
\argin{info}{Array of info structures (array of handles)}
\argin{ninfo}{Number of elements in the \refarg{info} array (integer)}
\argin{cbfunc}{Callback function \refapi{pmix_op_cbfunc_t} (function reference)}
\argin{cbdata}{Data to be passed to the callback function (memory reference)}
\end{arglist}

Returns one of the following:

\begin{itemize}
    \item \refconst{PMIX_SUCCESS}, indicating that the request is being processed by the host environment - result will be returned in the provided \refarg{cbfunc}. Note that the library must not invoke the callback function prior to returning from the \ac{API}.
    \item \refconst{PMIX_OPERATION_SUCCEEDED}, indicating that the request was immediately processed and returned \textit{success} - the \refarg{cbfunc} will \textit{not} be called.
    \item a PMIx error constant indicating either an error in the input or that the request was immediately processed and failed - the \refarg{cbfunc} will \textit{not} be called.
\end{itemize}

\reqattrstart
\ac{PMIx} libraries are not required to directly support any attributes for this function. However, any provided attributes must be passed to the host environment for processing, and the \ac{PMIx} library is required to add the \refAttributeItem{PMIX_USERID} and the \refAttributeItem{PMIX_GRPID} attributes of the client process that is requesting the operation.

\reqattrend

\optattrstart
The following attributes are optional for host environments that support this operation:

\pasteAttributeItem{PMIX_TIMEOUT}
\pasteAttributeItem{PMIX_RANGE}

\optattrend

%%%%
\descr

Non-blocking form of the \refapi{PMIx_Unpublish} function.
The callback function will be executed once the server confirms removal of the specified data. The \refarg{info} array must be maintained until the callback is provided.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%