[ExecuTorch][WebGPU] Repair dynamic SDPA routing, add attestation - #21134
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21134
Note: Links to docs will display an error until the docs builds have been completed. ❌ 2 New Failures, 35 PendingAs of commit 3d8e640 with merge base 28a7fac ( NEW FAILURES - The following jobs have failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
SS-JIA
left a comment
There was a problem hiding this comment.
Review automatically exported from Phabricator review in Meta.
fbsource master [ghstack-poisoned]
Pull Request resolved: #21134 With dynamic sequence positions, the backend could record only one SDPA route, so a decode following a dynamic-position prefill could dispatch the wrong kernel, and there was no way to confirm which kernel actually ran. This treats a dynamic SymInt position (not just a dynamic sequence dimension) as sufficient to record both SDPA routes, exposes active-kernel route attestation for correctness and performance harnesses, and reads the timestamp-query gate per execution so diagnostics can be enabled after module initialization. The attestation state and query are compiled only under the WGPU_BACKEND_ENABLE_PROFILING build flag, so production builds carry no additional state or cost, mirroring how the Vulkan backend gates its QueryPool behind ET_EVENT_TRACER_ENABLED. No Vulkan analogue for the routing fix (WebGPU runtime routing and observability). Key changes: - WebGPUGraph.{cpp,h}: expose the active-route attestation query (compile-gated behind WGPU_BACKEND_ENABLE_PROFILING) and read the timestamp-query gate per execution. - WebGPUUtils.h, Sdpa.cpp: treat a dynamic SymInt position (not only a dynamic sequence dimension) as sufficient to record both SDPA routes. ghstack-source-id: 411961454 @exported-using-ghexport Differential Revision: [D113171745](https://our.internmc.facebook.com/intern/diff/D113171745/)
Pull Request resolved: #21134 With dynamic sequence positions, the backend could record only one SDPA route, so a decode following a dynamic-position prefill could dispatch the wrong kernel, and there was no way to confirm which kernel actually ran. This treats a dynamic SymInt position (not just a dynamic sequence dimension) as sufficient to record both SDPA routes, exposes active-kernel route attestation for correctness and performance harnesses, and reads the timestamp-query gate per execution so diagnostics can be enabled after module initialization. The attestation state and query are compiled only under the WGPU_BACKEND_ENABLE_PROFILING build flag, so production builds carry no additional state or cost, mirroring how the Vulkan backend gates its QueryPool behind ET_EVENT_TRACER_ENABLED. No Vulkan analogue for the routing fix (WebGPU runtime routing and observability). Key changes: - WebGPUGraph.{cpp,h}: expose the active-route attestation query (compile-gated behind WGPU_BACKEND_ENABLE_PROFILING) and read the timestamp-query gate per execution. - WebGPUUtils.h, Sdpa.cpp: treat a dynamic SymInt position (not only a dynamic sequence dimension) as sufficient to record both SDPA routes. ghstack-source-id: 411961454 @exported-using-ghexport Differential Revision: [D113171745](https://our.internmc.facebook.com/intern/diff/D113171745/)
Pull Request resolved: #21134 With dynamic sequence positions, the backend could record only one SDPA route, so a decode following a dynamic-position prefill could dispatch the wrong kernel, and there was no way to confirm which kernel actually ran. This treats a dynamic SymInt position (not just a dynamic sequence dimension) as sufficient to record both SDPA routes, exposes active-kernel route attestation for correctness and performance harnesses, and reads the timestamp-query gate per execution so diagnostics can be enabled after module initialization. The attestation state and query are compiled only under the WGPU_BACKEND_ENABLE_PROFILING build flag, so production builds carry no additional state or cost, mirroring how the Vulkan backend gates its QueryPool behind ET_EVENT_TRACER_ENABLED. No Vulkan analogue for the routing fix (WebGPU runtime routing and observability). Key changes: - WebGPUGraph.{cpp,h}: expose the active-route attestation query (compile-gated behind WGPU_BACKEND_ENABLE_PROFILING) and read the timestamp-query gate per execution. - WebGPUUtils.h, Sdpa.cpp: treat a dynamic SymInt position (not only a dynamic sequence dimension) as sufficient to record both SDPA routes. ghstack-source-id: 411961454 @exported-using-ghexport Differential Revision: [D113171745](https://our.internmc.facebook.com/intern/diff/D113171745/)
Pull Request resolved: #21134 With dynamic sequence positions, the backend could record only one SDPA route, so a decode following a dynamic-position prefill could dispatch the wrong kernel, and there was no way to confirm which kernel actually ran. This treats a dynamic SymInt position (not just a dynamic sequence dimension) as sufficient to record both SDPA routes, exposes active-kernel route attestation for correctness and performance harnesses, and reads the timestamp-query gate per execution so diagnostics can be enabled after module initialization. The attestation state and query are compiled only under the WGPU_BACKEND_ENABLE_PROFILING build flag, so production builds carry no additional state or cost, mirroring how the Vulkan backend gates its QueryPool behind ET_EVENT_TRACER_ENABLED. No Vulkan analogue for the routing fix (WebGPU runtime routing and observability). Key changes: - WebGPUGraph.{cpp,h}: expose the active-route attestation query (compile-gated behind WGPU_BACKEND_ENABLE_PROFILING) and read the timestamp-query gate per execution. - WebGPUUtils.h, Sdpa.cpp: treat a dynamic SymInt position (not only a dynamic sequence dimension) as sufficient to record both SDPA routes. ghstack-source-id: 411961454 @exported-using-ghexport Differential Revision: [D113171745](https://our.internmc.facebook.com/intern/diff/D113171745/)
Pull Request resolved: #21134 With dynamic sequence positions, the backend could record only one SDPA route, so a decode following a dynamic-position prefill could dispatch the wrong kernel, and there was no way to confirm which kernel actually ran. This treats a dynamic SymInt position (not just a dynamic sequence dimension) as sufficient to record both SDPA routes, exposes active-kernel route attestation for correctness and performance harnesses, and reads the timestamp-query gate per execution so diagnostics can be enabled after module initialization. The attestation state and query are compiled only under the WGPU_BACKEND_ENABLE_PROFILING build flag, so production builds carry no additional state or cost, mirroring how the Vulkan backend gates its QueryPool behind ET_EVENT_TRACER_ENABLED. No Vulkan analogue for the routing fix (WebGPU runtime routing and observability). Key changes: - WebGPUGraph.{cpp,h}: expose the active-route attestation query (compile-gated behind WGPU_BACKEND_ENABLE_PROFILING) and read the timestamp-query gate per execution. - WebGPUUtils.h, Sdpa.cpp: treat a dynamic SymInt position (not only a dynamic sequence dimension) as sufficient to record both SDPA routes. ghstack-source-id: 411961454 @exported-using-ghexport Differential Revision: [D113171745](https://our.internmc.facebook.com/intern/diff/D113171745/)
Pull Request resolved: #21134 With dynamic sequence positions, the backend could record only one SDPA route, so a decode following a dynamic-position prefill could dispatch the wrong kernel, and there was no way to confirm which kernel actually ran. This treats a dynamic SymInt position (not just a dynamic sequence dimension) as sufficient to record both SDPA routes, exposes active-kernel route attestation for correctness and performance harnesses, and reads the timestamp-query gate per execution so diagnostics can be enabled after module initialization. The attestation state and query are compiled only under the WGPU_BACKEND_ENABLE_PROFILING build flag, so production builds carry no additional state or cost, mirroring how the Vulkan backend gates its QueryPool behind ET_EVENT_TRACER_ENABLED. No Vulkan analogue for the routing fix (WebGPU runtime routing and observability). Key changes: - WebGPUGraph.{cpp,h}: expose the active-route attestation query (compile-gated behind WGPU_BACKEND_ENABLE_PROFILING) and read the timestamp-query gate per execution. - WebGPUUtils.h, Sdpa.cpp: treat a dynamic SymInt position (not only a dynamic sequence dimension) as sufficient to record both SDPA routes. ghstack-source-id: 411961454 @exported-using-ghexport Differential Revision: [D113171745](https://our.internmc.facebook.com/intern/diff/D113171745/)
Pull Request resolved: #21134 With dynamic sequence positions, the backend could record only one SDPA route, so a decode following a dynamic-position prefill could dispatch the wrong kernel, and there was no way to confirm which kernel actually ran. This treats a dynamic SymInt position (not just a dynamic sequence dimension) as sufficient to record both SDPA routes, exposes active-kernel route attestation for correctness and performance harnesses, and reads the timestamp-query gate per execution so diagnostics can be enabled after module initialization. The attestation state and query are compiled only under the WGPU_BACKEND_ENABLE_PROFILING build flag, so production builds carry no additional state or cost, mirroring how the Vulkan backend gates its QueryPool behind ET_EVENT_TRACER_ENABLED. No Vulkan analogue for the routing fix (WebGPU runtime routing and observability). Key changes: - WebGPUGraph.{cpp,h}: expose the active-route attestation query (compile-gated behind WGPU_BACKEND_ENABLE_PROFILING) and read the timestamp-query gate per execution. - WebGPUUtils.h, Sdpa.cpp: treat a dynamic SymInt position (not only a dynamic sequence dimension) as sufficient to record both SDPA routes. ghstack-source-id: 411961454 @exported-using-ghexport Differential Revision: [D113171745](https://our.internmc.facebook.com/intern/diff/D113171745/)
Stack from ghstack (oldest at bottom):
With dynamic sequence positions, the backend could record only one SDPA route,
so a decode following a dynamic-position prefill could dispatch the wrong
kernel, and there was no way to confirm which kernel actually ran. This treats a
dynamic SymInt position (not just a dynamic sequence dimension) as sufficient to
record both SDPA routes, exposes active-kernel route attestation for correctness
and performance harnesses, and reads the timestamp-query gate per execution so
diagnostics can be enabled after module initialization. The attestation state
and query are compiled only under the WGPU_BACKEND_ENABLE_PROFILING build flag,
so production builds carry no additional state or cost, mirroring how the Vulkan
backend gates its QueryPool behind ET_EVENT_TRACER_ENABLED. No Vulkan analogue
for the routing fix (WebGPU runtime routing and observability).
Key changes:
behind WGPU_BACKEND_ENABLE_PROFILING) and read the timestamp-query gate per
execution.
sequence dimension) as sufficient to record both SDPA routes.
@exported-using-ghexport
Differential Revision: D113171745
Differential Revision: D113171745