bazelbuild / bazelbuild/bazel

a compatibility issue between Bazel + rules_rust + clang around that compiler flag.

Open
#29,272 1 comment 0 reactions 0 assignees View on GitHub
team-Rules-CPP type: bug untriaged
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the bug:

This week, our team had met a bazel bug, our repository's bazel version is 9.0.0, when we run bazel test in a new server, the problem had occurred! The errors are as follows

```
FLAGS = Some(-U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 -D_FORTIFY_SOURCE=1 -DNDEBUG -ffunction-sections -fdata-sections -fno-canonical-system-headers -Wno-builtin-macro-redefined -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted")
cargo:rerun-if-env-changed=CC_SHELL_ESCAPED_FLAGS
CC_SHELL_ESCAPED_FLAGS = None
cargo:rerun-if-env-changed=HOST_CFLAGS
HOST_CFLAGS = Some(-resource-dir=/home/zhengchungui/.cache/bazel/_bazel_zhengchungui/933c2f549343e099e3a6911b282cb71c/sandbox/linux-sandbox/16546/execroot/_main/bazel-out/k8-opt-ST-ec1a37510e22/bin/third_party/rs/ffmpeg/clang_resource_dir)
cargo:rerun-if-env-changed=CC_SHELL_ESCAPED_FLAGS
CC_SHELL_ESCAPED_FLAGS = None
cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
CFLAGS_x86_64_unknown_linux_gnu = Some(-resource-dir=/home/zhengchungui/.cache/bazel/_bazel_zhengchungui/933c2f549343e099e3a6911b282cb71c/sandbox/linux-sandbox/16546/execroot/_main/bazel-out/k8-opt-ST-ec1a37510e22/bin/third_party/rs/ffmpeg/clang_resource_dir)
cargo:rerun-if-env-changed=CC_SHELL_ESCAPED_FLAGS
CC_SHELL_ESCAPED_FLAGS = None
cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
CFLAGS_x86_64-unknown-linux-gnu = Some(-resource-dir=/home/zhengchungui/.cache/bazel/_bazel_zhengchungui/933c2f549343e099e3a6911b282cb71c/sandbox/linux-sandbox/16546/execroot/_main/bazel-out/k8-opt-ST-ec1a37510e22/bin/third_party/rs/ffmpeg/clang_resource_dir)
cargo:rerun-if-env-changed=CC_SHELL_ESCAPED_FLAGS
CC_SHELL_ESCAPED_FLAGS = None

--stderr:
clang: error: unknown argument: '-fno-canonical-system-headers'

thread 'main' (7) panicked at external/rules_rust++crate+crates__ffmpeg-sys-next-8.1.0/[build.rs:809](http://build.rs:809/):9:
Compile failed
stack backtrace:
0: 0x56d4727faf72 - ::fmt::hac1d885928ba8582
1: 0x56d47280d2b7 - core::fmt::write::h83ebb4d32483be9e
2: 0x56d4727caa96 - std::io::Write::write_fmt::ha6a1d6c1ea64b2d0
3: 0x56d4727d9489 - std::panicking::default_hook::{{closure}}::h8e9c4d1276f0925f
4: 0x56d4727d92e9 - std::panicking::default_hook::h2b2078d38b534dfb
5: 0x56d4727d977b - std::panicking::panic_with_hook::h39b739724e701bfd
6: 0x56d4727d919c - std::panicking::begin_panic::{{closure}}::hd6deae5584501d8d
7: 0x56d4727d5e99 - std::sys::backtrace::__rust_end_short_backtrace::hd42f29d9708777c9
8: 0x56d4727d9162 - std::panicking::begin_panic::h5f6ce5fb1e89204a
9: 0x56d4723fa858 - build_script_build::main::hbda0ab34faa44852
10: 0x56d47240d853 - std::sys::backtrace::__rust_begin_short_backtrace::hd1eee0d71eae11df
11: 0x56d47240d849 - std::rt::lang_start::{{closure}}::h5d0ac55d05e22b6d
12: 0x56d4727cc3c6 - std::rt::lang_start_internal::h74b643a2cc7fe3b4
13: 0x56d472407205 - main
14: 0x76d94ef6bd90 -
15: 0x76d94ef6be40 - __libc_start_main
16: 0x56d4723f16b5 - _start
17: 0x0 -

ERROR: /home/zhengchungui/.cache/bazel/_bazel_zhengchungui/933c2f549343e099e3a6911b282cb71c/external/rules_rust++crate+crates__ffmpeg-sys-next-8.1.0/BUILD.bazel:86:19: Running Cargo build script ffmpeg-sys-next failed: (Exit 1): runner failed: error executing CargoBuildScriptRun command (from cargo_build_script rule target @@rules_rust++crate+crates__ffmpeg-sys-next-8.1.0//:_bs) bazel-out/k8-opt-exec/bin/external/rules_rust+/cargo/private/cargo_build_script_runner/runner '--script=bazel-out/k8-fastbuild/bin/external/rules_rust++crate+crates__ffmpeg-sys-next-8.1.0/_bs-' ... (remaining 9 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
Build script process failed with exit code 101
--stdout:
cargo:rustc-link-search=native=/home/zhengchungui/.cache/bazel/_bazel_zhengchungui/933c2f549343e099e3a6911b282cb71c/sandbox/linux-sandbox/16561/execroot/_main/bazel-out/k8-fastbuild/bin/third_party/rs/ffmpeg/dist_dir/lib
cargo:rustc-link-lib=dylib=avcodec
cargo:rustc-link-lib=dylib=avdevice
cargo:rustc-link-lib=dylib=avfilter
cargo:rustc-link-lib=dylib=avformat
cargo:rustc-link-lib=dylib=avutil
cargo:rustc-link-lib=dylib=swresample
cargo:rustc-link-lib=dylib=swscale
cargo:rustc-link-arg=-Wl,--no-as-needed
cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-gnu
CC_x86_64-unknown-linux-gnu = Some(/home/zhengchungui/.cache/bazel/_bazel_zhengchungui/933c2f549343e099e3a6911b282cb71c/sandbox/linux-sandbox/16561/execroot/_main/bazel-out/k8-opt-exec-ST-21a06e18d48a/bin/external/llvm-project+/clang/clang)
cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
CRATE_CC_NO_DEFAULTS = None
cargo:rerun-if-env-changed=CFLAGS
CFLAGS = Some(-U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -fno-canonical-system-headers -Wno-builtin-macro-redefined -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted")
cargo:rerun-if-env-changed=CC_SHELL_ESCAPED_FLAGS
CC_SHELL_ESCAPED_FLAGS = None
cargo:rerun-if-env-changed=HOST_CFLAGS
HOST_CFLAGS = Some(-resource-dir=/home/zhengchungui/.cache/bazel/_bazel_zhengchungui/933c2f549343e099e3a6911b282cb71c/sandbox/linux-sandbox/16561/execroot/_main/bazel-out/k8-fastbuild/bin/third_party/rs/ffmpeg/clang_resource_dir)
cargo:rerun-if-env-changed=CC_SHELL_ESCAPED_FLAGS
CC_SHELL_ESCAPED_FLAGS = None
cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
CFLAGS_x86_64_unknown_linux_gnu = Some(-resource-dir=/home/zhengchungui/.cache/bazel/_bazel_zhengchungui/933c2f549343e099e3a6911b282cb71c/sandbox/linux-sandbox/16561/execroot/_main/bazel-out/k8-fastbuild/bin/third_party/rs/ffmpeg/clang_resource_dir)
cargo:rerun-if-env-changed=CC_SHELL_ESCAPED_FLAGS
CC_SHELL_ESCAPED_FLAGS = None
cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
CFLAGS_x86_64-unknown-linux-gnu = Some(-resource-dir=/home/zhengchungui/.cache/bazel/_bazel_zhengchungui/933c2f549343e099e3a6911b282cb71c/sandbox/linux-sandbox/16561/execroot/_main/bazel-out/k8-fastbuild/bin/third_party/rs/ffmpeg/clang_resource_dir)
cargo:rerun-if-env-changed=CC_SHELL_ESCAPED_FLAGS
CC_SHELL_ESCAPED_FLAGS = None

--stderr:
clang: error: unknown argument: '-fno-canonical-system-headers'

thread 'main' (7) panicked at external/rules_rust++crate+crates__ffmpeg-sys-next-8.1.0/[build.rs:809](http://build.rs:809/):9:
Compile failed
stack backtrace:
0: 0x63f71536cf72 - ::fmt::hac1d885928ba8582
1: 0x63f71537f2b7 - core::fmt::write::h83ebb4d32483be9e
2: 0x63f71533ca96 - std::io::Write::write_fmt::ha6a1d6c1ea64b2d0
3: 0x63f71534b489 - std::panicking::default_hook::{{closure}}::h8e9c4d1276f0925f
4: 0x63f71534b2e9 - std::panicking::default_hook::h2b2078d38b534dfb
5: 0x63f71534b77b - std::panicking::panic_with_hook::h39b739724e701bfd
6: 0x63f71534b19c - std::panicking::begin_panic::{{closure}}::hd6deae5584501d8d
7: 0x63f715347e99 - std::sys::backtrace::__rust_end_short_backtrace::hd42f29d9708777c9
8: 0x63f71534b162 - std::panicking::begin_panic::h5f6ce5fb1e89204a
9: 0x63f714f6c858 - build_script_build::main::hbda0ab34faa44852
10: 0x63f714f7f853 - std::sys::backtrace::__rust_begin_short_backtrace::hd1eee0d71eae11df
11: 0x63f714f7f849 - std::rt::lang_start::{{closure}}::h5d0ac55d05e22b6d
12: 0x63f71533e3c6 - std::rt::lang_start_internal::h74b643a2cc7fe3b4
13: 0x63f714f79205 - main
14: 0x7d8bdc7cdd90 -
15: 0x7d8bdc7cde40 - __libc_start_main
16: 0x63f714f636b5 - _start
17: 0x0 -

WARNING: errors encountered while analyzing target '//tools/ide:refresh_compile_commands', it will not be built.
Analysis failed
WARNING: errors encountered while analyzing target '//tools/ide:refresh_compile_commands.py', it will not be built.
Analysis failed
INFO: Build succeeded for only 320 of 433 top-level targets
INFO: Found 148 targets and 283 test targets...
INFO: Elapsed time: 204.341s, Critical Path: 65.59s
INFO: 38650 processes: 83 action cache hit, 15101 disk cache hit, 22143 internal, 1580 linux-sandbox, 4 local.
ERROR: Build did NOT complete successfully
//prod/appliance/infer/rknn/hw:hw__rockchip_rk3576_build_test FAILED TO BUILD
//prod/appliance/infer/tools/bench/hw:bench__rockchip_rk3576_build_test FAILED TO BUILD
//prod/appliance/infer/tools/bench/hw:hw__rockchip_rk3576_build_test FAILED TO BUILD
//prod/appliance/lineqc/tennis/core:core__linux_amd64_build_test FAILED TO BUILD
//prod/appliance/lineqc/tennis/core:core__linux_arm64_build_test FAILED TO BUILD
(Skipping other failed to build tests)

Executed 145 out of 283 tests: 193 tests pass, 88 fail to build, and 2 were skipped.
```

Details:

The failure happens when rules_rust runs cargo build.rs.
That build.rs sees CFLAGS that Bazel injects, which include -fno-canonical-system-headers.
The clang version you’re using doesn’t accept that flag, so the probe/compile step fails.
As a result, many targets show FAILED TO BUILD. This is essentially an upstream / build-environment mismatch.

when we try git bisect to find the bug, it's running successfully till all the commits going through, so it's definitely not our code problem, it's kinda bazel bug

### Which category does this issue belong to?

Rules API

### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

_No response_

### Which operating system are you running Bazel on?

ubuntu 22.04

### What is the output of `bazel info release`?

9.0.0

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

_No response_

### What's the output of `git remote get-url origin; git rev-parse HEAD` ?

```text

```

### If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

_No response_

### Have you found anything relevant by searching the web?

_No response_

### Any other information, logs, or outputs that you want to share?

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with the rules_rust Cargo build script runner and the failing external/rules_rust++crate+crates__ffmpeg-sys-next-8.1.0/build.rs:809 path. Reproduce on Ubuntu 22.04 with Bazel 9.0.0 and inspect how the injected CFLAGS reach clang, especially -fno-canonical-system-headers. Done means reducing the failure to a minimal example and determining the Bazel-side handling needed for the incompatible flag.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.