bazelbuild / bazelbuild/bazel

C++ cross compilation from Linux to Windows doesn't work

Open
#19,208 3 comments 1 reaction 0 assignees View on GitHub
P2 team-Remote-Exec type: bug
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

When defining a C++ toolchain, paths to the various tools (compiler, linker, etc) must be supplied to `cc_toolchain_config`. These paths are used as-is if absolute, and otherwise assumed to be [relative](https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/rules/cpp/CcToolchainFeatures.java;l=997;drc=cc335fd27e3f8b5a57f98328a67e874d28f4d558) to the package where the toolchain is defined.

However, the logic to determine whether a path is absolute is [host-dependent](https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/vfs/PathFragment.java;drc=5a52a72d168cb0326929e0eeb3e0cbafd8b5bbac), so a path like `C:/foo/bar` is considered relative on Linux (because it doesn't start with `/`) but absolute on Windows. The result is that a Bazel running on Linux will produce an incorrect command line for a C++ action executed on Windows:

```
action 'Compiling win_from_linux.cc'
...
Command Line: (exec third_party/toolchains/rbe_windows_bazel_5.2.0_vs2019/C:/VS/VC/Tools/MSVC/14.29.30133/bin/HostX64/x64/cl.exe \
...
win_from_linux.cc)
```

(In this example, the toolchain was defined in the `//third_party/toolchains/rbe_windows_bazel_5.2.0_vs2019` package, and the compiler path was supplied as `C:/VS/VC/Tools/MSVC/14.29.30133/bin/HostX64/x64/cl.exe`.)

The full repro is available at https://github.com/jtattermusch/win-from-linux-rbe-repro (thanks @jtattermusch!). You must use a Bazel built at or after 2f0948b.

I can think of a few solutions:

- Parameterize the `PathFragment` absolutization logic by execution platform; likely comes a huge blast radius, since all call sites must be audited and the execution platform might have to be plumbed into places where it's not currently accessible.
- Consider paths starting with a drive letter (`X:/...`) to be absolute even on non-Windows systems; technically a breaking change, since these are valid relative paths on Unix.
- Make it possible to pass "uninterpreted" paths into `cc_toolchain_config` (i.e., no absolutization is attempted).

Contributor guide

Open the contributing guide

Research direction

Start with the linked logic in src/main/java/com/google/devtools/build/lib/rules/cpp/CcToolchainFeatures.java and src/main/java/com/google/devtools/build/lib/vfs/PathFragment.java. Run the win-from-linux-rbe-repro with a Bazel built at or after 2f0948b to reproduce the malformed Windows compiler path. Done means C++ cross-compilation from Linux to Windows no longer prefixes the drive-letter path with the toolchain package.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
build-system, compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.