Support for Sequence Parallelism (SP) in the Multimodal Pipeline? #56
Description
I’m wondering whether you have plans to add Sequence Parallelism (SP) support to the multimodal branch (e.g., image-text or video-text models). If SP is already on the roadmap, could you share a rough timeline or milestone for when an initial release might be expected? Knowing this would help us decide whether to wait for the official implementation or prototype an internal patch.
Appreciate any insights you can provide, and please let me know if community testing or feedback would be useful once the feature reaches an alpha stage.
Thanks again for your great work!