Parm: Efficient Training of Large Sparsely-Activated Models with Dedicated Schedules

This code repository is for 'Parm: Efficient Training of Large Sparsely-Activated Models with Dedicated Schedules,' a paper accepted at INFOCOM 2024.

Name		Name	Last commit message	Last commit date
Megatron-LM @ 8aa4619		Megatron-LM @ 8aa4619
test_overlap_alltoall_and_allgather		test_overlap_alltoall_and_allgather
.gitignore		.gitignore
.gitmodules		.gitmodules
LICENSE		LICENSE
README.md		README.md

Navigation Menu