intel / intel/torch-xpu-ops

[nativert] Runtime has no XPU support: placement rejects XPU, no XPU AOTI runner, no SPIR-V Triton loader

Open
#5,364 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
113
Forks
128
Avg merge
5d 9h
Merged PRs (30d)
112

Description

`torch/nativert` cannot execute a PT2 archive placed on XPU. Three independent gaps, in the order an XPU model hits them (paths from pytorch/pytorch main @ 589c789f3a9):

1. `isSameDevice` (`torch/nativert/executor/PlacementUtils.cpp:5-25`) allowlists CPU/CUDA/meta/MTIA and does `TORCH_CHECK(false, "isSameDevice: Unsupported device type ...")` for anything else. It is reached for every weight from `torch/nativert/executor/Weights.cpp:176`, so loading a model with `xpu:0` tensor meta throws immediately. `Placement` itself is device-agnostic and `convertDevice` (`torch/nativert/graph/Graph.cpp:1167-1185`) parses `"xpu:0"` fine, so the rejection is purely this allowlist.

2. The AOTI delegate resolves its runner through `AOTIModelContainerRunnerRegistry()->Create(device.type(), ...)` (`torch/nativert/executor/AOTInductorDelegateExecutor.cpp:112-118`), and only two keys are ever registered: `at::kCPU` (`AOTInductorDelegateExecutor.cpp:72-74`) and `at::kCUDA` (`AOTInductorModelContainerCudaShim.cpp:20-22`, built from the CUDA-gated `libtorch_nativert_cuda_sources` list in `build_variables.bzl:675-678`). There is no XPU shim and no `libtorch_nativert_xpu_sources`, so `Create(kXPU, ...)` returns `nullptr` and the next line null-derefs without a `TORCH_CHECK`. An XPU AOTI runner already exists upstream (`torch/csrc/inductor/aoti_runner/model_container_runner_xpu.cpp`), so this is wiring rather than new functionality.

3. `TritonKernelManagerRegistry` dispatch (`torch/nativert/kernels/TritonKernel.cpp:215-241`) keys off the artifact extension: `.cubin` -> kCUDA, `.hsaco` -> kHIP, `.so` -> kCPU, `.bin` -> kMTIA. There is no `.spv`/SYCL branch, so an XPU Triton kernel falls through to `TORCH_CHECK(loader_ != nullptr, "couldn't find triton kernel loader ...")` at `TritonKernel.cpp:243-245`.

The Python side is already device-generic: `lower_exported_program` calls plain `torch._inductor.aot_compile` (`torch/nativert/backends/_lower_utils.py:56-58`). I confirmed on Intel GPU that export, AOTI lowering and `package_pt2` all succeed for an XPU module and the failure comes only from the runner. Note that in OSS builds nothing runs on any device yet: `PyModelRunner` is bound to an empty `StubModelRunner` behind `#ifdef FBCODE_CAFFE2` (`torch/nativert/python/Bindings.cpp:83-100`), which is why `test/export/test_nativert.py` is gated `FBcode only for now`.

Filed as the tracked reference for the `@skipXPUIf` on `TestNativeRTDevice.test_aoti` in `test/export/test_nativert.py`.

> Drafted with the assistance of an AI assistant and reviewed before filing.

Contributor guide

Open the contributing guide

Research direction

Start with PlacementUtils.cpp, AOTInductorDelegateExecutor.cpp, the CUDA shim and build_variables.bzl, then trace TritonKernel.cpp and the XPU runner in torch/csrc/inductor/aoti_runner. Run test/export/test_nativert.py and inspect the existing XPU skip; done means XPU placement, AOTI runner creation, and SPIR-V kernel loading no longer fail at those paths.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, github-actions, python, pytorch
Domain
backend, build-system, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.