TL/UCP: Add linear alltoall and allgather algorithms based on xgvmi ucp get #992
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nsarka
force-pushed
the
nsarka/allgather_xgvmi
branch
from
3b8cbf2
to
ecbd9e1
Compare
nsarka
force-pushed
the
nsarka/allgather_xgvmi
branch
from
ecbd9e1
to
a362467
Compare
nsarka
changed the title
janjust
approved these changes
Jun 26, 2024
|
I don't think we should do this now, but these algorithms, including sliding-window AR will not need the allgather in the init function when #909 is merged |
samnordmann
reviewed
Jul 1, 2024
| @@ -74,6 +75,9 @@ ucc_status_t ucc_tl_ucp_allgather_sparbit_init(ucc_base_coll_args_t *coll_args, | |||
| ucc_base_team_t *team, | |||
| ucc_coll_task_t **task_h); | |||
|
|
|||
| /* XGVMI */ | |||
| void ucc_tl_ucp_dpu_xgvmi_rdma_progress_allgather(ucc_coll_task_t *coll_task); | |||
There was a problem hiding this comment.
Suggested change
| void ucc_tl_ucp_dpu_xgvmi_rdma_progress_allgather(ucc_coll_task_t *coll_task); | |
| void ucc_tl_ucp_dpu_allgather_xgvmi_rdma_progress(ucc_coll_task_t *coll_task); |
to keep a pattern "tl_coll_algo_(init|start|progress|finalize)"
| @@ -0,0 +1,73 @@ | |||
| /** | |||
| * Copyright(c) 2021-2024, NVIDIA CORPORATION & AFFILIATES. All rights reserved. | |||
There was a problem hiding this comment.
nit:
Suggested change
| * Copyright(c) 2021-2024, NVIDIA CORPORATION & AFFILIATES. All rights reserved. | |
| * Copyright(c) 2024, NVIDIA CORPORATION & AFFILIATES. All rights reserved. |
|
|
||
| req_param.op_attr_mask |= UCP_OP_ATTR_FIELD_MEMH; | ||
|
|
||
| for (i = *posted; i < host_team_size; i++) { |
There was a problem hiding this comment.
is it possible that posted is not 0 when entering this function?
Would it make sense to put this first loop in a "start" function instead?
|
|
||
| ucp_worker_progress(tl_ctx->worker.ucp_worker); | ||
|
|
||
| for (i = *completed; i < *posted; i++) { |
There was a problem hiding this comment.
here *posted is necessarily equal to host_team_size right?
| @@ -14,6 +14,7 @@ enum { | |||
| UCC_TL_UCP_ALLTOALL_ALG_PAIRWISE, | |||
| UCC_TL_UCP_ALLTOALL_ALG_BRUCK, | |||
| UCC_TL_UCP_ALLTOALL_ALG_ONESIDED, | |||
| UCC_TL_UCP_ALLTOALL_ALG_LINEAR_XGVMI, | |||
There was a problem hiding this comment.
please fix copyright here, and check in other files as well
| @@ -136,6 +136,14 @@ typedef struct ucc_tl_ucp_task { | |||
| ucc_ee_executor_task_t *reduce_task; | |||
| ucc_tl_ucp_dpu_offload_buf_info_t *bufs; | |||
| } allreduce_sliding_window; | |||
| struct { | |||
| ucc_tl_ucp_allreduce_sw_host_allgather *allgather_data; | |||
| @@ -49,5 +49,22 @@ ucc_status_t ucc_tl_ucp_allreduce_sliding_window_register( | |||
| ucp_context_h ucp_context, ucc_tl_ucp_team_t *tl_team, | |||
| struct ucc_tl_ucp_allreduce_sw_export_buf *ebuf, void *packed_memh); | |||
|
|
|||
| void ucc_tl_ucp_dpu_xgvmi_free_rkeys( | |||
There was a problem hiding this comment.
Declaration could be removed from the header
| ucc_coll_task_t *coll_task); | ||
|
|
||
| ucc_status_t | ||
| ucc_tl_ucp_dpu_xgvmi_rdma_task_finalize( |
|
Can you update the tests as well? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is a follow up to allreduce sliding window. It adds linear alltoall and allgather algorithms based on XGVMI. They will post ucp gets from host to host in a round robin fashion.