[nativert] Runtime has no XPU support: placement rejects XPU, no XPU AOTI runner, no SPIR-V Triton loader
- Dominant language
- Python
- Stars
- 113
- Forks
- 128
- Avg merge
- 5d 9h
- Merged PRs (30d)
- 112
Description
`torch/nativert` cannot execute a PT2 archive placed on XPU. Three independent gaps, in the order an XPU model hits them (paths from pytorch/pytorch main @ 589c789f3a9):
1. `isSameDevice` (`torch/nativert/executor/PlacementUtils.cpp:5-25`) allowlists CPU/CUDA/meta/MTIA and does `TORCH_CHECK(false, "isSameDevice: Unsupported device type ...")` for anything else. It is reached for every weight from `torch/nativert/executor/Weights.cpp:176`, so loading a model with `xpu:0` tensor meta throws immediately. `Placement` itself is device-agnostic and `convertDevice` (`torch/nativert/graph/Graph.cpp:1167-1185`) parses `"xpu:0"` fine, so the rejection is purely this allowlist.
2. The AOTI delegate resolves its runner through `AOTIModelContainerRunnerRegistry()->Create(device.type(), ...)` (`torch/nativert/executor/AOTInductorDelegateExecutor.cpp:112-118`), and only two keys are ever registered: `at::kCPU` (`AOTInductorDelegateExecutor.cpp:72-74`) and `at::kCUDA` (`AOTInductorModelContainerCudaShim.cpp:20-22`, built from the CUDA-gated `libtorch_nativert_cuda_sources` list in `build_variables.bzl:675-678`). There is no XPU shim and no `libtorch_nativert_xpu_sources`, so `Create(kXPU, ...)` returns `nullptr` and the next line null-derefs without a `TORCH_CHECK`. An XPU AOTI runner already exists upstream (`torch/csrc/inductor/aoti_runner/model_container_runner_xpu.cpp`), so this is wiring rather than new functionality.
3. `TritonKernelManagerRegistry` dispatch (`torch/nativert/kernels/TritonKernel.cpp:215-241`) keys off the artifact extension: `.cubin` -> kCUDA, `.hsaco` -> kHIP, `.so` -> kCPU, `.bin` -> kMTIA. There is no `.spv`/SYCL branch, so an XPU Triton kernel falls through to `TORCH_CHECK(loader_ != nullptr, "couldn't find triton kernel loader ...")` at `TritonKernel.cpp:243-245`.
The Python side is already device-generic: `lower_exported_program` calls plain `torch._inductor.aot_compile` (`torch/nativert/backends/_lower_utils.py:56-58`). I confirmed on Intel GPU that export, AOTI lowering and `package_pt2` all succeed for an XPU module and the failure comes only from the runner. Note that in OSS builds nothing runs on any device yet: `PyModelRunner` is bound to an empty `StubModelRunner` behind `#ifdef FBCODE_CAFFE2` (`torch/nativert/python/Bindings.cpp:83-100`), which is why `test/export/test_nativert.py` is gated `FBcode only for now`.
Filed as the tracked reference for the `@skipXPUIf` on `TestNativeRTDevice.test_aoti` in `test/export/test_nativert.py`.
> Drafted with the assistance of an AI assistant and reviewed before filing.
Contributor guide
Research direction
Start with PlacementUtils.cpp, AOTInductorDelegateExecutor.cpp, the CUDA shim and build_variables.bzl, then trace TritonKernel.cpp and the XPU runner in torch/csrc/inductor/aoti_runner. Run test/export/test_nativert.py and inspect the existing XPU skip; done means XPU placement, AOTI runner creation, and SPIR-V kernel loading no longer fail at those paths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, github-actions, python, pytorch
- Domain
- backend, build-system, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100