feat: add an XPU graph backend

.claude/skills/torch-tweak-assistant/SKILL.md

@@ -71,6 +71,11 @@ Do NOT activate this skill when:
 - **Best for**: PyTorch compiler-based optimization on Intel GPU / XPU
 - **Simple**: `TorchInductorBackend()` with no config needed
 
+### XPU Graph Backend
+- **Best for**: replaying a module that was recorded once during tuning
+- **Simple**: `XPUGraphBackend()`
+- **Rejection**: failed, empty, or incorrect recordings are dropped for that module and input
+
 ### OpenVINO Backend
 - **Best for**: an OpenVINO model on an Intel CPU or GPU
 - **Simple**: `OpenVINOBackend()` with no config needed
@@ -93,8 +98,8 @@ Do NOT activate this skill when:
 ## Best Practices
 
 ### Backend Selection
-- **HighestThroughputStrategy**: TorchAO, Torch Inductor, OpenVINO, and eager. It measures each one and keeps the fastest. List order does not change the winner.
-- **FirstWinsStrategy**: default order is Torch Inductor, OpenVINO, then eager. The first backend that builds is kept. TorchAO is not in that default, because the first backend that builds wins and a quantization that builds would be kept without a speed comparison.
+- **HighestThroughputStrategy**: the default list is Torch Inductor, XPU graph, and eager. It measures each one and keeps the fastest. List order does not change the winner. TorchAO and OpenVINO are optional additions.
+- **FirstWinsStrategy**: default order is Torch Inductor, XPU graph, OpenVINO, then eager. The first backend that builds is kept. TorchAO is not in that default, because the first backend that builds wins and a quantization that builds would be kept without a speed comparison.
 
 ### Common Issues
 - **Graph breaks**: conditional logic in a module blocks tuning of that module
@@ -124,6 +129,7 @@ tt.load(model, "tuned_model.tt")
 from torch_tweak.torch.backend import (
     TorchAOBackend,
     TorchInductorBackend,
+    XPUGraphBackend,
     OpenVINOBackend,
     TorchEagerBackend,
 )
@@ -133,6 +139,7 @@ strategy = HighestThroughputStrategy(
     backends=[
         TorchAOBackend(),
         TorchInductorBackend(),
+        XPUGraphBackend(),
         OpenVINOBackend(),
         TorchEagerBackend(),
     ]

README.md

@@ -35,7 +35,7 @@ This page is the landing summary. The same file is the PyPI long description.
 | Feature                     | Description                                                                                                               |
 |-----------------------------|---------------------------------------------------------------------------------------------------------------------------|
 | Ease-of-use                 | Single line of code to run all possible tuning paths directly from your source code                                       |
-| Wide Backend Support        | Compatible with TorchAO, Torch Inductor, OpenVINO, and eager PyTorch                                                      |
+| Wide Backend Support        | Compatible with TorchAO, Torch Inductor, XPU graph, OpenVINO, and eager PyTorch                                           |
 | Model Tuning                | Enhance the performance of models such as ResNET and BERT for efficient inference deployment                              |
 | Pipeline Tuning             | Streamline Python code pipelines for models such as Stable Diffusion and Flux using seamless model wrapping and tuning    |
 | Correctness Testing         | Ensures tuned models produce correct outputs by validating on provided data samples                                       |
@@ -55,6 +55,7 @@ use frameworks like vLLM or SGLang for best performance. Use Torch Tweak for gen
 from torch_tweak.torch.backend import (
     TorchAOBackend,
     TorchInductorBackend,
+    XPUGraphBackend,
     OpenVINOBackend,
     TorchEagerBackend,
 )
@@ -65,13 +66,14 @@ strategy = HighestThroughputStrategy(
     backends=[
         TorchAOBackend(),
         TorchInductorBackend(),
+        XPUGraphBackend(),
         OpenVINOBackend(),
         TorchEagerBackend(),
     ]
 )
 ```
 
-TorchAO, Torch Inductor, and OpenVINO configuration is in the [backends guide](docs/learn/backends.md).
+TorchAO, Torch Inductor, XPU graph, and OpenVINO configuration is in the [backends guide](docs/learn/backends.md).
 Torch Eager accepts an optional autocast dtype and otherwise runs the module unchanged.
 Tune strategies are in [core functionalities](docs/learn/core_functionalities.md).
 

docs/api/backends/xpu_graph_backend.md

@@ -0,0 +1,20 @@
+<!--
+Copyright (c) 2026 Intel Corporation
+SPDX-License-Identifier: Apache-2.0
+-->
+
+# XPU Graph Backend API
+
+## XPUGraphBackend
+
+::: torch_tweak.torch.backend.XPUGraphBackend
+    options:
+      show_root_heading: true
+      heading_level: 3
+
+## XPUGraphBackendConfig
+
+::: torch_tweak.torch.backend.XPUGraphBackendConfig
+    options:
+      show_root_heading: true
+      heading_level: 3

docs/guides/backends/xpu_graph_backend.md

@@ -0,0 +1,35 @@
+<!--
+Copyright (c) 2026 Intel Corporation
+SPDX-License-Identifier: Apache-2.0
+-->
+
+# XPU Graph Backend Guide
+
+The XPU graph backend records the module on an Intel GPU once during tuning and replays that recording on later inference calls.
+Recording is not repeated for each inference call.
+
+## Overview
+
+- **One recording**: warmup and capture run in `build`, while the module is being tuned
+- **Replay**: inference copies new tensor values into the recorded inputs and calls `replay`
+- **Rejection**: a failed recording, an empty recording, or outputs that disagree with eager drop this backend for that module and input
+- **Static shapes**: a batch axis or another dynamic axis rejects the backend. Replay cannot change tensor shapes
+
+## Quick Start
+
+```python
+from torch_tweak.torch.backend import XPUGraphBackend
+from torch_tweak.torch.tune_strategy import HighestThroughputStrategy
+
+strategy = HighestThroughputStrategy(backends=[XPUGraphBackend()])
+```
+
+The backend is in the default `FirstWinsStrategy` and `HighestThroughputStrategy` lists, directly after Torch Inductor.
+
+PyTorch records a native SYCL graph when it was built with a SYCL compiler of 2026.1 or newer, except on PVC.
+An older build records the legacy SYCL command graph.
+This backend uses the recording that build already performs.
+
+## Next Steps
+
+- API reference: [XPU Graph Backend](../../api/backends/xpu_graph_backend.md)

docs/guides/tune_strategies/tune_strategies.md

@@ -81,7 +81,7 @@ tt.tune(model, input_data)
 
 ## FirstWinsStrategy
 
-Tries backends in priority order and returns the first one that successfully builds and validates. If a backend fails, the strategy moves on to the next candidate instead of aborting. If all backends fail, the error is caught and the original model is used as-is. The default order is Torch Inductor, then OpenVINO, then eager. Put a backend earlier only when you want that one to win as soon as it builds.
+Tries backends in priority order and returns the first one that successfully builds and validates. If a backend fails, the strategy moves on to the next candidate instead of aborting. If all backends fail, the error is caught and the original model is used as-is. The default order is Torch Inductor, XPU graph, OpenVINO, then eager. Put a backend earlier only when you want that one to win as soon as it builds.
 
 ### Usage
 

docs/learn/backends.md

@@ -33,6 +33,20 @@ backend = TorchInductorBackend()
 
 Guide: [Torch Inductor Backend](../guides/backends/torch_inductor_backend.md).
 
+## XPU Graph Backend
+
+The XPU graph backend records the module once while tuning and replays that recording for later inference.
+PyTorch chooses native or legacy recording. This backend uses whichever one that build records.
+A recording that fails, or that disagrees with eager, is rejected for that module and input.
+
+```python
+from torch_tweak.torch.backend import XPUGraphBackend
+
+backend = XPUGraphBackend()
+```
+
+Guide: [XPU Graph Backend](../guides/backends/xpu_graph_backend.md).
+
 ## OpenVINO Backend
 
 The OpenVINO backend converts the module to an OpenVINO model and runs it on an Intel CPU or GPU.

docs/learn/core_functionalities.md

@@ -111,6 +111,7 @@ from torch_tweak.torch.tune_strategy import FirstWinsStrategy
 strategy = FirstWinsStrategy(
     backends=[
         TorchInductorBackend(),
+        XPUGraphBackend(),
         OpenVINOBackend(),
         TorchEagerBackend(),
     ]
@@ -131,7 +132,7 @@ strategy = OneBackendStrategy(backend=OpenVINOBackend())
 
 ## HighestThroughputStrategy
 
-Profiles all compatible backends and selects the fastest. Use this when maximum throughput matters and you can afford longer tuning time. `HighestThroughputStrategy()` with no backend list compares Torch Inductor and eager only.
+Profiles all compatible backends and selects the fastest. Use this when maximum throughput matters and you can afford longer tuning time. `HighestThroughputStrategy()` with no backend list compares Torch Inductor, XPU graph, and eager.
 
 ```python
 from torch_tweak.torch.tune_strategy import HighestThroughputStrategy

docs/learn/overview.md

@@ -12,7 +12,7 @@ NOTE: This file has been modified by Intel Corporation.
 
 **Torch Tweak** is an inference toolkit designed for tuning and deploying Deep Learning models with a focus on Intel XPU (GPU) accelerators. It provides model tuning capabilities through compilation and conversion paths that can significantly improve inference speed and efficiency across various AI workloads including Computer Vision, Natural Language Processing, Speech Recognition, and Generative AI.
 
-The toolkit enables seamless tuning of PyTorch models and pipelines using TorchAO, Torch Inductor, OpenVINO, and eager through a single Python API. The resulting tuned models are ready for deployment in production environments.
+The toolkit enables seamless tuning of PyTorch models and pipelines using TorchAO, Torch Inductor, XPU graph, OpenVINO, and eager through a single Python API. The resulting tuned models are ready for deployment in production environments.
 
 Torch Tweak works with your environment - relying first on your software versions - and selects the best-performing backend for your software and hardware setup, guiding you to supported technologies.
 
@@ -31,7 +31,7 @@ The distinct capabilities of Torch Tweak are summarized in the feature matrix:
 | Feature                     | Description                                                                                                               |
 |-----------------------------|---------------------------------------------------------------------------------------------------------------------------|
 | Ease-of-use                 | Single line of code to run all possible tuning paths directly from your source code                                       |
-| Wide Backend Support        | Compatible with TorchAO, Torch Inductor, OpenVINO, and eager                                                              |
+| Wide Backend Support        | Compatible with TorchAO, Torch Inductor, XPU graph, OpenVINO, and eager                                                   |
 | Model Tuning                | Enhance the performance of models such as ResNET and BERT for efficient inference deployment                              |
 | Pipeline Tuning             | Streamline Python code pipelines for models such as Stable Diffusion and Flux using seamless model wrapping and tuning    |
 | Correctness Testing         | Ensures tuned models produce correct outputs by validating on provided data samples                                       |

examples/E5Large/e5large/tune.py

@@ -11,7 +11,11 @@
 
 import torch
 from torch_tweak.torch import HighestThroughputStrategy, inspect, save, tune, wrap
-from torch_tweak.torch.backend import TorchEagerBackend, TorchInductorBackend
+from torch_tweak.torch.backend import (
+    TorchEagerBackend,
+    TorchInductorBackend,
+    XPUGraphBackend,
+)
 from torch_tweak.torch.config import config as global_config
 
 from .cmd_args import get_parser
@@ -97,6 +101,7 @@ def call_wrapper(*args, **kwargs):
             backends=[
                 TorchEagerBackend(),
                 TorchInductorBackend(),
+                XPUGraphBackend(),
             ]
         ).enable_find_max_batch_size(True),
     )

examples/ESM2/esm2/tune.py

@@ -11,7 +11,11 @@
 from transformers import AutoTokenizer, EsmForMaskedLM
 
 import torch_tweak.torch as tt
-from torch_tweak.torch.backend import TorchEagerBackend, TorchInductorBackend
+from torch_tweak.torch.backend import (
+    TorchEagerBackend,
+    TorchInductorBackend,
+    XPUGraphBackend,
+)
 
 DEVICE = torch.device("xpu")
 MODEL_NAME = "facebook/esm2_t33_650M_UR50D"
@@ -61,6 +65,7 @@ def tune(
         strategy = tt.FirstWinsStrategy(
             backends=[
                 TorchInductorBackend(),
+                XPUGraphBackend(),
                 TorchEagerBackend(),
             ]
         )

examples/FLUX/flux/tune.py

@@ -17,6 +17,7 @@
     OpenVINOBackendConfig,
     TorchEagerBackend,
     TorchInductorBackend,
+    XPUGraphBackend,
 )
 
 logger = getLogger(__name__)
@@ -58,6 +59,7 @@ def tune_model(
         strategy = tt.FirstWinsStrategy(
             backends=[
                 TorchInductorBackend(),
+                XPUGraphBackend(),
                 OpenVINOBackend(
                     OpenVINOBackendConfig(
                         use_dynamo=False,

examples/Minimal/demo.py

@@ -9,7 +9,11 @@
 import torch.nn as nn
 
 import torch_tweak.torch as tt
-from torch_tweak.torch.backend import TorchEagerBackend, TorchInductorBackend
+from torch_tweak.torch.backend import (
+    TorchEagerBackend,
+    TorchInductorBackend,
+    XPUGraphBackend,
+)
 
 
 def main() -> None:
@@ -19,13 +23,17 @@ def main() -> None:
     model = nn.Sequential(nn.Linear(32, 32), nn.ReLU(), nn.Linear(32, 8)).eval().to(device)
     sample = torch.randn(32, device=device)
 
-    # Skip the max-batch search. The two batch sizes below are enough to pick a backend.
+    # One batch size keeps every tensor shape static, so the XPU graph can be recorded.
     strategy = tt.HighestThroughputStrategy(
-        backends=[TorchInductorBackend(), TorchEagerBackend()],
+        backends=[
+            TorchInductorBackend(),
+            XPUGraphBackend(),
+            TorchEagerBackend(),
+        ],
     ).enable_find_max_batch_size(False)
 
     module = tt.Module(model, name="mlp", strategy=strategy)
-    tt.tune(module, sample, batch_sizes=[1, 4], device=device)
+    tt.tune(module, sample, batch_sizes=[4], device=device)
 
     results = strategy.results[-1].highest_throughput_results if strategy.results else []
     if not results:

examples/ParakeetCTC/parakeet_ctc/tune.py

@@ -15,7 +15,11 @@
 from parakeet_ctc.cmd_args import parse_args
 from parakeet_ctc.model import get_model
 from torch_tweak.torch import FirstWinsStrategy, TuneStrategy, inspect, save, tune, wrap
-from torch_tweak.torch.backend import TorchEagerBackend, TorchInductorBackend
+from torch_tweak.torch.backend import (
+    TorchEagerBackend,
+    TorchInductorBackend,
+    XPUGraphBackend,
+)
 
 logger = getLogger(__name__)
 
@@ -81,6 +85,7 @@ def main():
     strategy = FirstWinsStrategy(
         backends=[
             TorchInductorBackend(),
+            XPUGraphBackend(),
             TorchEagerBackend(),
         ]
     )

examples/ParakeetRNNT/parakeet_rnnt/tune.py

@@ -19,6 +19,7 @@
     TorchEagerBackend,
     TorchInductorBackend,
     TorchInductorBackendConfig,
+    XPUGraphBackend,
 )
 
 logger = getLogger(__name__)
@@ -48,6 +49,7 @@ def tune_model(
     strategy = strategy or HighestThroughputStrategy(
         backends=[
             TorchInductorBackend(TorchInductorBackendConfig(autocast_enabled=True, autocast_dtype=torch.float16)),
+            XPUGraphBackend(),
             TorchEagerBackend(),
         ]
     ).enable_find_max_batch_size(False)

examples/ResNet/resnet/tune.py

@@ -20,6 +20,7 @@
     TorchEagerBackend,
     TorchInductorBackend,
     TorchInductorBackendConfig,
+    XPUGraphBackend,
 )
 
 logger = getLogger(__name__)
@@ -60,6 +61,7 @@ def tune_model(
                 TorchInductorBackend(
                     config=TorchInductorBackendConfig(autocast_enabled=True, autocast_dtype=torch.float16)
                 ),
+                XPUGraphBackend(),
                 TorchEagerBackend(),
             ]
         ).enable_find_max_batch_size(False),

examples/StableDiffusion/stable_diffusion/tuning_strategy.py

@@ -13,6 +13,7 @@
     TorchEagerBackend,
     TorchInductorBackend,
     TorchInductorBackendConfig,
+    XPUGraphBackend,
 )
 from torch_tweak.torch.tune_strategy import HighestThroughputStrategy, OneBackendStrategy
 
@@ -35,6 +36,7 @@ def _inductor_backend() -> TorchInductorBackend:
 def _backend_catalog() -> dict[str, Backend]:
     return {
         "inductor": _inductor_backend(),
+        "xpu_graph": XPUGraphBackend(),
         "openvino": _openvino_backend(use_dynamo=False),
         "openvino_dynamo": _openvino_backend(use_dynamo=True),
         "eager": TorchEagerBackend(),

properdocs.yml

@@ -85,6 +85,7 @@ nav:
                 - OpenVINO Backend: guides/backends/openvino_backend.md
                 - TorchAO Backend: guides/backends/torchao_backend.md
                 - Torch Inductor Backend: guides/backends/torch_inductor_backend.md
+                - XPU Graph Backend: guides/backends/xpu_graph_backend.md
                 - Torch Eager Backend: guides/backends/torch_eager_backend.md
             - Tune Strategies: guides/tune_strategies/tune_strategies.md
             - Deployment: guides/deployment/deployment.md
@@ -114,6 +115,7 @@ nav:
                     -   OpenVINO Backend: api/backends/openvino_backend.md
                     -   TorchAO Backend: api/backends/torchao_backend.md
                     -   Torch Inductor Backend: api/backends/torch_inductor_backend.md
+                    -   XPU Graph Backend: api/backends/xpu_graph_backend.md
                     -   Torch Eager Backend: api/backends/torch_eager_backend.md
             -   Tuning Strategies API:
                     -   FirstWinsStrategy: api/tune_strategies/first_wins_strategy.md