[ExecuTorch][WebGPU] Add HuggingFace rotate-half RoPE - #21135
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21135
Note: Links to docs will display an error until the docs builds have been completed. ❌ 2 New Failures, 29 PendingAs of commit 2372deb with merge base 28a7fac ( NEW FAILURES - The following jobs have failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
SS-JIA
left a comment
There was a problem hiding this comment.
Review automatically exported from Phabricator review in Meta.
fbsource master [ghstack-poisoned]
Pull Request resolved: #21135 **Add HuggingFace rotate-half RoPE and migrate RoPE to shared dispatch construction** Qwen models pair the first and second halves of each head vector rather than adjacent elements. This adds the rotate-half operator with dynamic start-position updates, full-dimension Q/K handling, generated WGSL, and strict frequency-table bounds. The shared RoPE handler also closes the dispatch-boilerplate review: it uses `graph.device()`, typed graph-owned parameter buffers, descriptor-driven bindings and pipelines, named validation and resize callbacks, generated shader-registry lookup, scoped ownership, and graph-owned workgroup recomputation. Interleaved shader math and output are unchanged. Key changes: - `rotary_embedding_hf.wgsl` and generated registry entry — one thread per pair for HuggingFace rotate-half. - `RotaryEmbedding.cpp` — named validation, typed resize contexts, shared dispatch descriptors, and one initial/resize grid picker per route. - Native tests — malformed input rejection, dynamic bounds, shrink/regrow reuse, and HF/interleaved lifecycle coverage. Co-authored-with: Claude Code. ghstack-source-id: 411961455 @exported-using-ghexport Differential Revision: [D113171746](https://our.internmc.facebook.com/intern/diff/D113171746/)
Pull Request resolved: #21135 **Add HuggingFace rotate-half RoPE and migrate RoPE to shared dispatch construction** Qwen models pair the first and second halves of each head vector rather than adjacent elements. This adds the rotate-half operator with dynamic start-position updates, full-dimension Q/K handling, generated WGSL, and strict frequency-table bounds. The shared RoPE handler also closes the dispatch-boilerplate review: it uses `graph.device()`, typed graph-owned parameter buffers, descriptor-driven bindings and pipelines, named validation and resize callbacks, generated shader-registry lookup, scoped ownership, and graph-owned workgroup recomputation. Interleaved shader math and output are unchanged. Key changes: - `rotary_embedding_hf.wgsl` and generated registry entry — one thread per pair for HuggingFace rotate-half. - `RotaryEmbedding.cpp` — named validation, typed resize contexts, shared dispatch descriptors, and one initial/resize grid picker per route. - Native tests — malformed input rejection, dynamic bounds, shrink/regrow reuse, and HF/interleaved lifecycle coverage. Co-authored-with: Claude Code. ghstack-source-id: 411961455 @exported-using-ghexport Differential Revision: [D113171746](https://our.internmc.facebook.com/intern/diff/D113171746/)
Pull Request resolved: #21135 **Add HuggingFace rotate-half RoPE and migrate RoPE to shared dispatch construction** Qwen models pair the first and second halves of each head vector rather than adjacent elements. This adds the rotate-half operator with dynamic start-position updates, full-dimension Q/K handling, generated WGSL, and strict frequency-table bounds. The shared RoPE handler also closes the dispatch-boilerplate review: it uses `graph.device()`, typed graph-owned parameter buffers, descriptor-driven bindings and pipelines, named validation and resize callbacks, generated shader-registry lookup, scoped ownership, and graph-owned workgroup recomputation. Interleaved shader math and output are unchanged. Key changes: - `rotary_embedding_hf.wgsl` and generated registry entry — one thread per pair for HuggingFace rotate-half. - `RotaryEmbedding.cpp` — named validation, typed resize contexts, shared dispatch descriptors, and one initial/resize grid picker per route. - Native tests — malformed input rejection, dynamic bounds, shrink/regrow reuse, and HF/interleaved lifecycle coverage. Co-authored-with: Claude Code. ghstack-source-id: 411961455 @exported-using-ghexport Differential Revision: [D113171746](https://our.internmc.facebook.com/intern/diff/D113171746/)
Pull Request resolved: #21135 **Add HuggingFace rotate-half RoPE and migrate RoPE to shared dispatch construction** Qwen models pair the first and second halves of each head vector rather than adjacent elements. This adds the rotate-half operator with dynamic start-position updates, full-dimension Q/K handling, generated WGSL, and strict frequency-table bounds. The shared RoPE handler also closes the dispatch-boilerplate review: it uses `graph.device()`, typed graph-owned parameter buffers, descriptor-driven bindings and pipelines, named validation and resize callbacks, generated shader-registry lookup, scoped ownership, and graph-owned workgroup recomputation. Interleaved shader math and output are unchanged. Key changes: - `rotary_embedding_hf.wgsl` and generated registry entry — one thread per pair for HuggingFace rotate-half. - `RotaryEmbedding.cpp` — named validation, typed resize contexts, shared dispatch descriptors, and one initial/resize grid picker per route. - Native tests — malformed input rejection, dynamic bounds, shrink/regrow reuse, and HF/interleaved lifecycle coverage. Co-authored-with: Claude Code. ghstack-source-id: 411961455 @exported-using-ghexport Differential Revision: [D113171746](https://our.internmc.facebook.com/intern/diff/D113171746/)
Pull Request resolved: #21135 **Add HuggingFace rotate-half RoPE and migrate RoPE to shared dispatch construction** Qwen models pair the first and second halves of each head vector rather than adjacent elements. This adds the rotate-half operator with dynamic start-position updates, full-dimension Q/K handling, generated WGSL, and strict frequency-table bounds. The shared RoPE handler also closes the dispatch-boilerplate review: it uses `graph.device()`, typed graph-owned parameter buffers, descriptor-driven bindings and pipelines, named validation and resize callbacks, generated shader-registry lookup, scoped ownership, and graph-owned workgroup recomputation. Interleaved shader math and output are unchanged. Key changes: - `rotary_embedding_hf.wgsl` and generated registry entry — one thread per pair for HuggingFace rotate-half. - `RotaryEmbedding.cpp` — named validation, typed resize contexts, shared dispatch descriptors, and one initial/resize grid picker per route. - Native tests — malformed input rejection, dynamic bounds, shrink/regrow reuse, and HF/interleaved lifecycle coverage. Co-authored-with: Claude Code. ghstack-source-id: 411961455 @exported-using-ghexport Differential Revision: [D113171746](https://our.internmc.facebook.com/intern/diff/D113171746/)
Pull Request resolved: #21135 **Add HuggingFace rotate-half RoPE and migrate RoPE to shared dispatch construction** Qwen models pair the first and second halves of each head vector rather than adjacent elements. This adds the rotate-half operator with dynamic start-position updates, full-dimension Q/K handling, generated WGSL, and strict frequency-table bounds. The shared RoPE handler also closes the dispatch-boilerplate review: it uses `graph.device()`, typed graph-owned parameter buffers, descriptor-driven bindings and pipelines, named validation and resize callbacks, generated shader-registry lookup, scoped ownership, and graph-owned workgroup recomputation. Interleaved shader math and output are unchanged. Key changes: - `rotary_embedding_hf.wgsl` and generated registry entry — one thread per pair for HuggingFace rotate-half. - `RotaryEmbedding.cpp` — named validation, typed resize contexts, shared dispatch descriptors, and one initial/resize grid picker per route. - Native tests — malformed input rejection, dynamic bounds, shrink/regrow reuse, and HF/interleaved lifecycle coverage. Co-authored-with: Claude Code. ghstack-source-id: 411961455 @exported-using-ghexport Differential Revision: [D113171746](https://our.internmc.facebook.com/intern/diff/D113171746/)
Pull Request resolved: #21135 **Add HuggingFace rotate-half RoPE and migrate RoPE to shared dispatch construction** Qwen models pair the first and second halves of each head vector rather than adjacent elements. This adds the rotate-half operator with dynamic start-position updates, full-dimension Q/K handling, generated WGSL, and strict frequency-table bounds. The shared RoPE handler also closes the dispatch-boilerplate review: it uses `graph.device()`, typed graph-owned parameter buffers, descriptor-driven bindings and pipelines, named validation and resize callbacks, generated shader-registry lookup, scoped ownership, and graph-owned workgroup recomputation. Interleaved shader math and output are unchanged. Key changes: - `rotary_embedding_hf.wgsl` and generated registry entry — one thread per pair for HuggingFace rotate-half. - `RotaryEmbedding.cpp` — named validation, typed resize contexts, shared dispatch descriptors, and one initial/resize grid picker per route. - Native tests — malformed input rejection, dynamic bounds, shrink/regrow reuse, and HF/interleaved lifecycle coverage. Co-authored-with: Claude Code. ghstack-source-id: 411961455 @exported-using-ghexport Differential Revision: [D113171746](https://our.internmc.facebook.com/intern/diff/D113171746/)
Pull Request resolved: #21135 **Add HuggingFace rotate-half RoPE and migrate RoPE to shared dispatch construction** Qwen models pair the first and second halves of each head vector rather than adjacent elements. This adds the rotate-half operator with dynamic start-position updates, full-dimension Q/K handling, generated WGSL, and strict frequency-table bounds. The shared RoPE handler also closes the dispatch-boilerplate review: it uses `graph.device()`, typed graph-owned parameter buffers, descriptor-driven bindings and pipelines, named validation and resize callbacks, generated shader-registry lookup, scoped ownership, and graph-owned workgroup recomputation. Interleaved shader math and output are unchanged. Key changes: - `rotary_embedding_hf.wgsl` and generated registry entry — one thread per pair for HuggingFace rotate-half. - `RotaryEmbedding.cpp` — named validation, typed resize contexts, shared dispatch descriptors, and one initial/resize grid picker per route. - Native tests — malformed input rejection, dynamic bounds, shrink/regrow reuse, and HF/interleaved lifecycle coverage. Co-authored-with: Claude Code. ghstack-source-id: 411961455 @exported-using-ghexport Differential Revision: [D113171746](https://our.internmc.facebook.com/intern/diff/D113171746/)
Stack from ghstack (oldest at bottom):
Add HuggingFace rotate-half RoPE and migrate RoPE to shared dispatch construction
Qwen models pair the first and second halves of each head vector rather than adjacent elements. This adds the rotate-half operator with dynamic start-position updates, full-dimension Q/K handling, generated WGSL, and strict frequency-table bounds.
The shared RoPE handler also closes the dispatch-boilerplate review: it uses
graph.device(), typed graph-owned parameter buffers, descriptor-driven bindings and pipelines, named validation and resize callbacks, generated shader-registry lookup, scoped ownership, and graph-owned workgroup recomputation. Interleaved shader math and output are unchanged.Key changes:
rotary_embedding_hf.wgsland generated registry entry — one thread per pair for HuggingFace rotate-half.RotaryEmbedding.cpp— named validation, typed resize contexts, shared dispatch descriptors, and one initial/resize grid picker per route.Co-authored-with: Claude Code.
@exported-using-ghexport
Differential Revision: D113171746
Differential Revision: D113171746