torch.roll in DeepSpeed

[sarvghotra]

Hi,

By any chance, does DeepSpeed have an optimized implementation (CUDA) for torch.roll function?

If not, can you please provide some pointers? Context: I am trying to implement cyclic shift in Shifted Window-Multi head Self Attention (SW-MSA) from Swin Transformer paper by reusing DeepSpeed's kernel code here.