Qwen3 MoE Support

[SonicSaurav]

Are MoEs supported out of the box? If not, what’s the recommended way to create a custom MoE configuration? Any guidance would be appreciated

[wwwjn]

For Qwen3 MoE, we have plan to work on it, but it's not high priority for us now. MoEs are not supported out of the box, you'll need to change model.py to add MoE parts.