Skip to content

Add Intel GPU (xpu) support via PyTorch's native xpu backend - #362

Open
frobnitzem wants to merge 1 commit into
hexgrad:mainfrom
frobnitzem:xpu-support
Open

Add Intel GPU (xpu) support via PyTorch's native xpu backend#362
frobnitzem wants to merge 1 commit into
hexgrad:mainfrom
frobnitzem:xpu-support

Conversation

@frobnitzem

Copy link
Copy Markdown

Summary

Adds 'xpu' to KPipeline's existing device auto-detect chain
(cudaxpumpscpu), using PyTorch's native torch.xpu backend
(built in since torch 2.5 — no intel-extension-for-pytorch dependency).
Packages it as an opt-in xpu extra alongside a new cpu extra so the
default install experience (pip install kokoro, no extras) is unchanged.

New install is selectable:

pip install kokoro # defaults to torch package (pulls in cuda)
pip install kokoro[cpu]
pip install kokoro[xpu]

Why

Kokoro currently only auto-detects cuda/mps, leaving Intel integrated
and discrete GPUs on the CPU path. That's a real, currently-idle install
base — every recent Intel laptop chip (Meteor Lake and newer) ships an iGPU
capable of this. No prior Intel-GPU support exists in this repo, and no
prior work landed it upstream here either — see "Prior art" below; every
real result found lives in a downstream fork or wasn't published as code.

Prior art (see NOTES.md for the full scan with sources)

  • Multiple independent projects report 3–5x (OpenVINO EP: Unicorn-Orator,
    magicunicorn) to 16–151x (native PyTorch xpu: crunchtools, unpublished)
    speedups on Intel iGPU vs CPU, on chip families including this one's
    (Meteor Lake).
  • No prior Intel NPU support was found working for Kokoro anywhere —
    ruled out for this PR - nobody's published a derivative model that uses NPU yet.
    It may not be the right fit for NPU anyway.
  • No prior work modifies hexgrad/kokoro itself; all real numbers found
    live in downstream wrappers (Kokoro-FastAPI, Unicorn-Orator, OpenArc)
    or were never published as code (crunchtools' patch is described in prose
    only). This PR is, as far as this scan found, the first attempt to land
    Intel GPU support in the upstream package directly.

What changed

  • kokoro/pipeline.py: KPipeline.__init__'s device auto-detect chain gains
    an xpu branch, gated by hasattr(torch, 'xpu') and torch.xpu.is_available()

  • pyproject.toml: adds cpu and xpu as mutually-exclusive
    [project.optional-dependencies] (via [tool.uv] conflicts), each
    routing torch to the appropriate PyTorch wheel index
    (download.pytorch.org/whl/{cpu,xpu}, both explicit = true). Base
    dependencies still lists bare torch, unchanged, for plain
    pip install kokoro.

A packaging gotcha worth flagging explicitly

First attempt kept torch unconstrained in base dependencies (as today)
and added a single opt-in xpu extra layered on top, routed via
[tool.uv.sources] with an extra == "xpu" marker — the pattern that looks
most natural for "don't touch the default, just add an opt-in". This
broke the default install: uv sync with no extra requested silently
installed torch==2.13.0+xpu anyway. Reproduced directly in this repo
(commit history on this branch before the fix). Root cause, best
understanding: uv's resolver treats the bare unconstrained torch
requirement as satisfiable by any valid source once one is introduced
anywhere in the dependency graph, including the xpu-indexed wheel — it
finds one universal answer that satisfies every extra-fork simultaneously
rather than forking per-extra, since nothing forces the two requirements
apart.

Fix: follow the pattern used by uv's own multi-backend docs and by
an example pyproject.toml known to work
(cpu/cuda/rocm as explicit, [tool.uv] conflicts-declared mutually
exclusive extras) — add cpu as a real extra alongside xpu, forcing uv to
actually fork the resolution instead of collapsing it. This is a slightly
different shape
than "layer an accelerator extra on an unconstrained
base dependency" — worth a maintainer opinion on whether cpu/xpu-as-
parallel-extras is the right long-term shape here, or whether the project
would rather see this solved a different way (e.g. documenting a manual
--index override instead of shipping extras at all). Flagging rather than
silently picking one, since it's a design call, not just a bugfix.

Measured 4.4x speedup cpu ~> xpu with this fix (excluding warmup round).

This is a same-shape, single-sentence, steady-state number, not a
line-for-line reproduction of the larger third-party figures cited above
(different text lengths per call reintroduce shape-recompile overhead —
not measured here, worth a follow-up if torch.compile/shape-caching
becomes part of this PR)

Test plan

  • uv sync (no extras) still installs plain torch from the default
    index — confirmed torch==2.13.0+cu130 (no xpu suffix) on this
    Linux/x86_64 box, i.e. default behavior unchanged from main.
  • uv sync --extra xpu installs torch==2.13.0+xpu,
    torch.xpu.is_available() returns True.
  • KPipeline(lang_code='a', device='xpu') runs end-to-end (real
    hexgrad/Kokoro-82M weights, real voice pack) and produces audio.
  • Benchmarked CPU vs XPU on identical input, see Results above.
  • Existing tests/test_custom_stft.py suite run on both baseline
    main (unmodified, torch+cpu) and this branch (torch+xpu):
    identical result both times — 2 passed, 2 failed. The 2 failures
    (test_stft_reconstruction, test_different_window_sizes) are a
    pre-existing CustomSTFT bug on main, reproduced verbatim before
    this patch touches anything; this PR introduces zero new
    failures and changes no test's outcome. (Note: pytest isn't a
    declared dependency of this project — installed locally via
    uv add --dev pytest to run the suite, not part of this diff.)
  • Not yet run: device='mps' / device='cuda' regression check (no
    Apple/NVIDIA hardware available in this environment to test on;
    reasoning: the new xpu_available check is computed once and only
    read inside the device is None branch and the new device == 'xpu'
    guard, neither of which touches the existing cuda/mps branches).

- KPipeline's device auto-detect chain gains 'xpu' (cuda -> xpu -> mps -> cpu),
  using torch.xpu.is_available() -- no intel-extension-for-pytorch dependency,
  since IPEX is being retired (maintenance-only through March 2026) and
  torch's own xpu backend has been built in since 2.5.
- Packaging: cpu/xpu declared as mutually-exclusive optional-dependencies
  (uv's documented multi-backend-torch pattern)

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant