RFC: Linux→Windows cross-build + Wine lit suite for offloader

Background

I've been bringing up PSO-based raytracing tests on Linux (#1268 / #1270 / #1273 / #1275 / #1277 / #1278 / #1279) and got the offloader cross-building to Windows MSVC ABI and running under Wine + vkd3d-proton against a real DXR runtime — useful enough that the foundational test passed end-to-end on an NVIDIA RTX 3060 without dual-booting or a Windows CI runner in the loop. Want to gauge whether wiring this into the project's CMake + lit infrastructure is something the maintainers would accept upstream.

What "this" is

Linux host (clang as cross-compiler, lld-link, llvm-rc) plus a Windows SDK and CRT acquired by the user → cross-built offloader.exe → executed by Wine. With binfmt_misc configured (wine-binfmt on Arch, etc.) the .exe runs transparently from a shell or execve. Notable side effect: the Windows-binary offloader can target either D3D12 (via vkd3d-proton) or Vulkan (Windows vulkan-1.dll → Wine → host Linux Vulkan loader), so the same suite covers both APIs against one cross-built binary. That also makes it a useful differential test against the native Linux Vulkan offloader — same .test files, two configurations.

Why it could be worth in-tree

• Linux contributors working on the D3D12 path get a real DXR loop without a Windows VM or dual-boot.
• Cheaper / faster pre-merge gate — Linux GPU runners are noticeably cheaper than Windows GPU runners. Wine + vkd3d-proton isn't bit-exact native D3D12, but it catches the obvious regressions before the slow Windows runner does.
• Mesa / vkd3d-proton contributors already run this exact stack; an in-tree CMake target would give them a DXR conformance harness with zero setup beyond a normal LLVM build.
• clang-dxc + offload-test-suite differential signal — once clang-dxc lowers [shader("…")] RT entry points, the same .test file can be validated through both native Linux Vulkan and Windows-binary-via-Wine, which is a useful sanity check on both the compile path and the runtime path.

What it'd require in-tree

Three small pieces, none of which is invasive:

• A toolchain file (something like cmake/toolchains/windows-msvc.cmake) that takes user-provided paths for WINDOWS_SDK_INCLUDE_DIRS, WINDOWS_SDK_LIB_DIR, WINDOWS_CRT_INCLUDE_DIR, WINDOWS_CRT_LIB_DIR. SDK-source agnostic — xwin, msvc-wine, EWDK, hand-copied from a Windows machine all work. Toolchain only knows how to point clang / lld-link at whatever paths the caller fills in.
• A wine lit suite (parallel to clang-vk / clang-d3d12 / clang-mtl) generated when OFFLOADTEST_WINE_OFFLOADER is set. The substitution is the bare offloader.exe path; an OFFLOADTEST_WINE_PREFIX_COMMAND cache var (default empty) lets users without binfmt set it to wine explicitly.
• A short docs page listing two-stage configure (native build + cross-build pointed at the toolchain) and the three or four well-known ways to get the Windows SDK headers, without preferring any.

The lit suite itself is almost identical to clang-d3d12's — only the offloader path differs.

Choices to make / questions for the team

1. Take it at all? Whether "Linux cross-builds Windows binary and runs it through Wine" is a workflow the project wants to bless. Reasonable arguments either way — extra surface to maintain vs. real iteration-speed win for Linux contributors.
2. One suite or two? Single wine suite covering both D3D12 + Vulkan (let the test's --api selection / SPIR-V-magic-vs-DXBC-prefix auto-detect route per file), or separate wine-d3d12 and wine-vk suites. The single-suite version is closer to how clang-vk works today and avoids generating two near-duplicate lit configs.
3. Configure flow. Two-stage configure (user runs cmake -B build && cmake -B build-win -DCMAKE_TOOLCHAIN_FILE=… && cmake --build build-win --target offloader) and the main build picks up the artifact — simpler PR. Or ExternalProject_Add-driven sub-build that hides the two stages — fancier but more CMake magic. I'd prefer two-stage for review tractability.
4. Binfmt or prefix. Default OFFLOADTEST_WINE_PREFIX_COMMAND to wine (works everywhere) or to empty (relies on the user's binfmt setup, slightly cleaner)? Either way the escape hatch is the same; the question is which default surprises fewer people.
5. CI integration. Whether the project wants a Linux + Wine + vkd3d-proton GitHub Actions runner alongside the existing Windows / macOS runners, or whether this stays a contributor-iteration tool only.

Gotchas I hit while bringing this up

For reference, in case any of this informs the toolchain file's defaults:

• Use -Xclang -nostdsysteminc, not -nostdinc. The latter also strips clang's builtin headers — xmmintrin.h / emmintrin.h / etc. disappear, and libpng's SSE2-intrinsic SIMD code links against undefined _mm_* symbols.
• CMAKE_MSVC_RUNTIME_LIBRARY=MultiThreadedDLL. xwin doesn't ship msvcrtd.lib, so the default debug runtime breaks the configure-time try_compile.
• Don't set CMAKE_AR to llvm-lib. CMake's MSVC mode auto-picks the right /OUT:-style invocation; if CMAKE_AR is explicit, CMake falls back to ar qc style and llvm-lib rejects it.
• lld-link flags need -Xlinker /libpath:… when invoked via clang — bare /libpath: is interpreted as an input file path by clang's driver.
• The host find_package(Vulkan) will succeed on Linux and pull /usr/include into the include path, shadowing xwin's CRT headers. Set CMAKE_DISABLE_FIND_PACKAGE_Vulkan=TRUE (or just disable the Vulkan backend) on the cross-build.
• find_package(D3D12) keys off Win10 SDK registry paths; the cross-build needs D3D12_INCLUDE_DIRS / D3D12_LIBRARIES passed explicitly.
• PNG_INTEL_SSE=off on libpng. Its hand-rolled SIMD wrappers emit external _mm_* symbols that lld-link can't resolve, even with the intrinsic headers reachable.

Happy to draft the toolchain file + lit suite + docs as a single small PR if there's interest. Wanted to probe the appetite before sending it.

cc @llvm-beanz

First read the existing clang-vk and clang-d3d12 lit suites and the project's CMake configuration. The RFC still needs a maintainer decision on scope, suite layout, configure flow, and CI; done would be an accepted design followed by the proposed Windows toolchain, Wine lit integration, and documentation.