[SYCL] Feature request: a compiler option to produce linker-agnostic object files from SYCL sources (-fsycl-link)
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
**Motivation**
In large-scale, heterogeneous projects like PyTorch, it is common to have a primary toolchain (e.g., GCC) for the main build, while specialized sources (like SYCL kernels) are compiled with a dedicated compiler (Clang/DPC++). The current challenge using SYCL enabled Clang lies in integrating the output of the SYCL compiler back into the main build pipeline, especially when the final linking stage is handled by a standard, SYCL-unaware linker.
Our build architecture for the PyTorch XPU backend is as follows:
1. **SYCL Compilation**: SYCL source files (`.cpp`) are compiled into object files (`.o`) using the SYCL compiler. These are then archived into a static library (e.g., `libtorch-xpu-ops.a`).
2. **Standard Compilation**: The rest of the PyTorch C++ sources are compiled using the standard system compiler (GCC).
3. **Final Linkage**: The SYCL static library (`libtorch-xpu-ops.a`) is linked with all other object files and libraries to produce the final shared library (`libtorch-xpu.so`). This final link step is performed by the standard, SYCL-unaware linker.
This separation is crucial because forcing a massive project like PyTorch to switch its primary toolchain to a SYCL-aware one is not feasible. It introduces significant build complexity, validation risks, and would face strong resistance from the upstream community, which rightly expects to build the project with a single, consistent toolchain.
**Problem Statement**
The standard build process using current version of SYCL enabled Clang breaks down at the final linkage stage. A SYCL-unaware linker cannot correctly process the device-specific sections and metadata from the object files generated by the SYCL compiler. This forces the final link step to also use a SYCL-aware toolchain, which violates our project's core architectural constraint.
We need a robust mechanism to compile a SYCL source file into an object file that encapsulates all necessary device code linkage internally. This "linker-agnostic" object file could then be treated like any other standard object file by any linker.
**Feature Request**
Can a new compiler flag, please, introduced which would instruct the SYCL compiler to perform the device code linking phase and embed the fully linked device binary into the output object file. On the example of the Intel's DPC++ compiler, such option is named **`-fsycl-link`**, see https://www.intel.com/content/www/us/en/docs/dpcpp-cpp-compiler/developer-guide-reference/2025-2/fsycl-link.html, and is used to build PyTorch with the Intel DPC++ compiler.
When this flag is used during the compilation of a SYCL source file (`kernel.cpp`) into an object file (`kernel.o`), the compiler would be responsible for:
1. Compiling the host and device code.
2. Linking the device code and resolving all its dependencies.
3. Embedding the final device image into a standard section of the resulting `kernel.o` file.
The resulting `kernel.o` would be a standard object file from the perspective of a SYCL-unaware toolchain, as all SYCL-specific linking would have been handled and self-contained. This would allow it to be seamlessly archived into a static library and subsequently linked by a standard linker without any issues.
While the exact implementation could vary, the key outcome is that building a SYCL-aware `.cpp` file into a SYCL-unaware `.o` file should be possible, whether via one or two command-line invocations for the compilation and device code linking. The critical part is that the creation of the final object file must handle the device code linkage.
This approach aligns with the principle of toolchain consistency in large projects and provides a clean, maintainable path for integrating specialized SYCL code into conventional build systems.
CC: @tahonermann @bader @YuriPlyakhin
Contributor guide
Research direction
The issue names no implementation files or tests. Start by tracing the SYCL compiler's existing device-linking and object-emission paths, then compare the requested -fsycl-link behavior with the referenced DPC++ option. Done means a SYCL source can produce an object containing its linked device image that a SYCL-unaware linker can consume.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- build-system, compilers
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100