[WIP]Sparse seqparallel#9

Open

zkh2016 wants to merge 8 commits intoPaddlePaddle:mainfrom

zkh2016:sparse_seqparallel

Commits on May 26, 2023

Commits on May 27, 2023

support bf16 for block
kuizhiqing
committed

Commits on May 29, 2023

fix bf16 bug in grad
kuizhiqing
committed

Commits on May 31, 2023

add dim128 support for block sparsewq
kuizhiqing
committed

Commits on Jun 5, 2023

Commits on Jun 7, 2023

support seqlen parallel
zhangkaihuo
committed