C++ header validation fails for MSVC remote compile
- Dominant language
- Java
- Stars
- 25.8k
- Forks
- 4.6k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 72
Description
### Description of the bug:
When building a `cc_library` that includes header files on Windows using an MSVC toolchain with remote execution, I get this error:
```
ERROR: C:/users/jay/code/engflow-example/cpp/BUILD:1:10: Compiling cpp/main.cc failed: undeclared inclusion(s) in rule '//cpp:cpp':
this rule is missing dependency declarations for the following files included by 'cpp/main.cc':
'C:/_b/gjo2whp5/execroot/ws/cpp/hello.h'
```
### Which category does this issue belong to?
C++/Objective-C Rules
### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
You will need a remote execution service with Windows that can execute actions in containers. I think that makes this difficult to reproduce, though maybe Google has something internally that might work? I'm working on this at EngFlow.
Assuming you have that:
1. Checkout https://github.com/EngFlow/example/pull/243
2. `bazel build --config=remote_windows_x64 //cpp:cpp_lib` (add `--remote_executor` and any other necessary flags).
### Which operating system are you running Bazel on?
Windows Server 2022
### What is the output of `bazel info release`?
release 6.3.2
### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.
n/a
### What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?
```text
n/a
```
### Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.
No.
### Have you found anything relevant by searching the web?
Issue [#9172](https://github.com/bazelbuild/bazel/issues/9172) - Windows: Bazel cannot share cpp cache between two projects with different workspace name
PR [#9188](https://github.com/bazelbuild/bazel/pull/9188) - Windows: Ignore workspace name differences while doing header checking
PR [#13339](https://github.com/bazelbuild/bazel/pull/13339) - Remote: Use execRoot as input root and do NOT set working directory by default.
### Any other information, logs, or outputs that you want to share?
I spent the morning poking through the source code to figure out how this works. It looks like Bazel's `CppCompile` action is trying to validate that 1) headers included from `srcs` are in either `srcs` or direct dependencies' `hdrs`, and 2) headers are not included with absolute paths.
For MSVC, Bazel accomplishes that by enabling the `/showincludes` flag, which tells MSVC to print the path of each included file. The paths are absolute. The full output from one of these actions looks like this:
hello.cc
Note: including file: C:\worker\work\2\execroot\ws\cpp/hello.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\string
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\yvals_core.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\vcruntime.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\sal.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\concurrencysal.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\vadefs.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\xkeycheck.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\xstring
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\iosfwd
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\yvals.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\crtdbg.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\vcruntime_new_debug.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\vcruntime_new.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\crtdefs.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\use_ansi.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\cstdio
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\stdio.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_wstdio.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_stdio_config.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\cstring
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\string.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_memory.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_memcpy_s.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\errno.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\vcruntime_string.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_wstring.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\cwchar
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\wchar.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_wconio.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_wctype.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_wdirect.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_wio.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_share.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_wprocess.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_wstdlib.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_wtime.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\sys/stat.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\sys/types.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\xstddef
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\cstddef
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\stddef.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\xtr1common
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\cstdlib
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\math.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_math.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\stdlib.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_malloc.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_search.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\limits.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\initializer_list
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\xmemory
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\cstdint
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\stdint.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\limits
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\cfloat
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\float.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\climits
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\intrin0.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\intrin0.inl.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\isa_availability.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\new
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\exception
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\type_traits
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\malloc.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\vcruntime_exception.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\eh.h
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\corecrt_terminate.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\xatomic.h
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\xutility
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\utility
Note: including file: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include\cctype
Note: including file: C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\ctype.h
This line is the one that matters:
```
Note: including file: C:\worker\work\2\execroot\ws\cpp/hello.h
```
This is not hermetic. The path of the execroot could be anything. Here's it's `C:\worker\work\2\execroot\ws`. Normally it would be `C:\worker\work\2\exec`, but I've hacked it in this build to try and work around the issue (no luck).
In [ShowIncludeFilter](https://github.com/bazelbuild/bazel/blob/c24b847322e7e21de10a9031aaae19a691475909/src/main/java/com/google/devtools/build/lib/rules/cpp/ShowIncludesFilter.java#L40), Bazel uses a regular expression to replace everything up to `.*execroot\\` with `../`, so that turns into `../ws/cpp/hello.h`. Later, that's joined with the local execroot within the workspace, so `C:/_b/gjo2whp5/execroot/ws/cpp/hello.h`, which is the path we see in the error message.
Bazel expects the path to be `C:/_b/gjo2whp5/execroot/example/cpp/hello.h` (`example` is the name of the workspace). Because it does not match, Bazel reports an error.
Tagging @meteorcloudy and @coeuvre who I think might have ideas on how to fix this. I'd be happy to work on this, but I'll need some guidance on what the solution should be and whether there's a good way to test it.
Contributor guide
Research direction
Start by tracing Bazel's CppCompile action and its MSVC /showincludes header-validation handling, focusing on how remote execution maps absolute execroot paths. Reproduce with the EngFlow example PR and the stated remote Windows build command; done means the declared cpp/hello.h dependency is recognized without relying on the worker's absolute path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100