Cross self attention switch#251

Open

entrpn wants to merge 10 commits intomainfrom

cross_self_attention_switch

Commits on Oct 10, 2025

skip flash block sizes setting for cross attention.

entrpn
authored and
coolkp
committed
change sharding based on cross/self attention.

entrpn
authored and
coolkp
committed
update sharding rules for attn.

entrpn
authored and
coolkp
committed
lint.

entrpn
authored and
coolkp
committed
ring attention rules are added at front if not present to shard sequence on fsdp axis
coolkp
committed
test fix
coolkp
committed

Commits on Oct 20, 2025

Add dense padded attention kernel and use unsafe rng key for generation
coolkp
committed

Commits on Oct 22, 2025