Add Intel GPU (xpu) support via PyTorch's native xpu backend - #362
Open
frobnitzem wants to merge 1 commit into
Open
Add Intel GPU (xpu) support via PyTorch's native xpu backend#362frobnitzem wants to merge 1 commit into
frobnitzem wants to merge 1 commit into
Conversation
- KPipeline's device auto-detect chain gains 'xpu' (cuda -> xpu -> mps -> cpu), using torch.xpu.is_available() -- no intel-extension-for-pytorch dependency, since IPEX is being retired (maintenance-only through March 2026) and torch's own xpu backend has been built in since 2.5. - Packaging: cpu/xpu declared as mutually-exclusive optional-dependencies (uv's documented multi-backend-torch pattern) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
'xpu'toKPipeline's existing device auto-detect chain(
cuda→xpu→mps→cpu), using PyTorch's nativetorch.xpubackend(built in since torch 2.5 — no
intel-extension-for-pytorchdependency).Packages it as an opt-in
xpuextra alongside a newcpuextra so thedefault install experience (
pip install kokoro, no extras) is unchanged.New install is selectable:
Why
Kokoro currently only auto-detects
cuda/mps, leaving Intel integratedand discrete GPUs on the CPU path. That's a real, currently-idle install
base — every recent Intel laptop chip (Meteor Lake and newer) ships an iGPU
capable of this. No prior Intel-GPU support exists in this repo, and no
prior work landed it upstream here either — see "Prior art" below; every
real result found lives in a downstream fork or wasn't published as code.
Prior art (see
NOTES.mdfor the full scan with sources)magicunicorn) to 16–151x (native PyTorch
xpu: crunchtools, unpublished)speedups on Intel iGPU vs CPU, on chip families including this one's
(Meteor Lake).
ruled out for this PR - nobody's published a derivative model that uses NPU yet.
It may not be the right fit for NPU anyway.
hexgrad/kokoroitself; all real numbers foundlive in downstream wrappers (
Kokoro-FastAPI,Unicorn-Orator, OpenArc)or were never published as code (crunchtools' patch is described in prose
only). This PR is, as far as this scan found, the first attempt to land
Intel GPU support in the upstream package directly.
What changed
kokoro/pipeline.py:KPipeline.__init__'s device auto-detect chain gainsan
xpubranch, gated byhasattr(torch, 'xpu') and torch.xpu.is_available()pyproject.toml: addscpuandxpuas mutually-exclusive[project.optional-dependencies](via[tool.uv] conflicts), eachrouting
torchto the appropriate PyTorch wheel index(
download.pytorch.org/whl/{cpu,xpu}, bothexplicit = true). Basedependenciesstill lists baretorch, unchanged, for plainpip install kokoro.A packaging gotcha worth flagging explicitly
First attempt kept
torchunconstrained in basedependencies(as today)and added a single opt-in
xpuextra layered on top, routed via[tool.uv.sources]with anextra == "xpu"marker — the pattern that looksmost natural for "don't touch the default, just add an opt-in". This
broke the default install:
uv syncwith no extra requested silentlyinstalled
torch==2.13.0+xpuanyway. Reproduced directly in this repo(commit history on this branch before the fix). Root cause, best
understanding: uv's resolver treats the bare unconstrained
torchrequirement as satisfiable by any valid source once one is introduced
anywhere in the dependency graph, including the xpu-indexed wheel — it
finds one universal answer that satisfies every extra-fork simultaneously
rather than forking per-extra, since nothing forces the two requirements
apart.
Fix: follow the pattern used by uv's own multi-backend docs and by
an example pyproject.toml known to work
(
cpu/cuda/rocmas explicit,[tool.uv] conflicts-declared mutuallyexclusive extras) — add
cpuas a real extra alongsidexpu, forcing uv toactually fork the resolution instead of collapsing it. This is a slightly
different shape than "layer an accelerator extra on an unconstrained
base dependency" — worth a maintainer opinion on whether
cpu/xpu-as-parallel-extras is the right long-term shape here, or whether the project
would rather see this solved a different way (e.g. documenting a manual
--indexoverride instead of shipping extras at all). Flagging rather thansilently picking one, since it's a design call, not just a bugfix.
Measured 4.4x speedup cpu
~>xpu with this fix (excluding warmup round).This is a same-shape, single-sentence, steady-state number, not a
line-for-line reproduction of the larger third-party figures cited above
(different text lengths per call reintroduce shape-recompile overhead —
not measured here, worth a follow-up if
torch.compile/shape-cachingbecomes part of this PR)
Test plan
uv sync(no extras) still installs plaintorchfrom the defaultindex — confirmed
torch==2.13.0+cu130(noxpusuffix) on thisLinux/x86_64 box, i.e. default behavior unchanged from
main.uv sync --extra xpuinstallstorch==2.13.0+xpu,torch.xpu.is_available()returnsTrue.KPipeline(lang_code='a', device='xpu')runs end-to-end (realhexgrad/Kokoro-82Mweights, real voice pack) and produces audio.tests/test_custom_stft.pysuite run on both baselinemain(unmodified,torch+cpu) and this branch (torch+xpu):identical result both times — 2 passed, 2 failed. The 2 failures
(
test_stft_reconstruction,test_different_window_sizes) are apre-existing
CustomSTFTbug onmain, reproduced verbatim beforethis patch touches anything; this PR introduces zero new
failures and changes no test's outcome. (Note:
pytestisn't adeclared dependency of this project — installed locally via
uv add --dev pytestto run the suite, not part of this diff.)device='mps'/device='cuda'regression check (noApple/NVIDIA hardware available in this environment to test on;
reasoning: the new
xpu_availablecheck is computed once and onlyread inside the
device is Nonebranch and the newdevice == 'xpu'guard, neither of which touches the existing
cuda/mpsbranches).