bazelbuild / bazelbuild/bazel

Bazel build missing dep declaration with transitive dependency

Open
#15,359 18 comments 5 reactions 1 assignee Claimed by @meteorcloudy View on GitHub
P2 team-Rules-CPP type: bug
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the bug:

I am building tensorflow project (commit: `2c6d3ed00f16838831aa460c5668a8466b9f3649`) and running into errors about the missing dependency declarations.

For instance, here is one of the errors:
```
ERROR: /usr/local/google/home/yeounoh/.cache/bazel/_bazel_yeounoh/dc4cfe365eb5fc5c1bdcf9c9346373b9/external/llvm-project/mlir/BUILD.bazel:3073:11: Compiling mlir/lib/Support/IndentedOstream.cpp failed: undeclared inclusion(s) in rule '@llvm-project//mlir:Support':
this rule is missing dependency declarations for the following files included by 'mlir/lib/Support/IndentedOstream.cpp':
'bazel-out/k8-opt-exec-50AE0418/bin/external/llvm-project/llvm/config.cppmap'
'bazel-out/k8-opt-exec-50AE0418/bin/external/llvm-project/llvm/Demangle.cppmap'
'bazel-out/k8-opt-exec-50AE0418/bin/external/llvm_terminfo/terminfo.cppmap'
'bazel-out/k8-opt-exec-50AE0418/bin/external/llvm_zlib/zlib.cppmap'
```

And here is the corresponding build def (from the build cache):
```
# /usr/local/google/home/yeounoh/.cache/bazel/_bazel_yeounoh/dc4cfe365eb5fc5c1bdcf9c9346373b9/external/llvm-project/mlir/BUILD.bazel:3073:11
cc_library(
name = "Support",
deps = ["@llvm-project//llvm:Support"],
includes = ["include"],
srcs = ["@llvm-project//mlir:lib/Support/DebugCounter.cpp", "@llvm-project//mlir:lib/Support/FileUtilities.cpp", "@llvm-project//mlir:lib/Support/IndentedOstream.cpp", "@llvm-project//mlir:lib/Support/InterfaceSupport.cpp", "@llvm-project//mlir:lib/Support/StorageUniquer.cpp", "@llvm-project//mlir:lib/Support/Timing.cpp", "@llvm-project//mlir:lib/Support/ToolUtilities.cpp", "@llvm-project//mlir:lib/Support/TypeID.cpp"],
hdrs = ["@llvm-project//mlir:include/mlir/Support/DebugAction.h", "@llvm-project//mlir:include/mlir/Support/DebugCounter.h", "@llvm-project//mlir:include/mlir/Support/DebugStringHelper.h", "@llvm-project//mlir:include/mlir/Support/FileUtilities.h", "@llvm-project//mlir:include/mlir/Support/IndentedOstream.h", "@llvm-project//mlir:include/mlir/Support/InterfaceSupport.h", "@llvm-project//mlir:include/mlir/Support/LLVM.h", "@llvm-project//mlir:include/mlir/Support/LogicalResult.h", "@llvm-project//mlir:include/mlir/Support/MathExtras.h", "@llvm-project//mlir:include/mlir/Support/StorageUniquer.h", "@llvm-project//mlir:include/mlir/Support/ThreadLocalCache.h", "@llvm-project//mlir:include/mlir/Support/Timing.h", "@llvm-project//mlir:include/mlir/Support/ToolUtilities.h", "@llvm-project//mlir:include/mlir/Support/TypeID.h"],
)
# Rule Support instantiated at (most recent call last):
# /usr/local/google/home/yeounoh/.cache/bazel/_bazel_yeounoh/dc4cfe365eb5fc5c1bdcf9c9346373b9/external/llvm-project/mlir/BUILD.bazel:3073:11 in
```

It doesn't include the missing depndencies, but just `@llvm-project/llvm:Support`; however, the build def of `@llvm-project/llvm:Support` does contain the missing dependency declarations (so it built successfully, too):
```
# /usr/local/google/home/yeounoh/.cache/bazel/_bazel_yeounoh/dc4cfe365eb5fc5c1bdcf9c9346373b9/external/llvm-project/llvm/BUILD.bazel:181:11
cc_library(
name = "Support",
deps = ["@llvm-project//llvm:config", "@llvm-project//llvm:Demangle", "@llvm_terminfo//:terminfo", "@llvm_zlib//:zlib"],
...
... (there is a long list of other build attributes)
```
If I manually add the missing deps directly to the `@llvm-project/mlirSupport` build def, then I can make it work (but it will run into other similar issues; repeat). I think there is something wrong with my setting that prevents transitive dependency.

### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Checkout https://github.com/tensorflow/tensorflow.git, commit: `2c6d3ed00f16838831aa460c5668a8466b9f3649`.

Try:
`bazel build //tensorflow/tools/pip_package:build_pip_package`

I asked my colleagues to try and some have and some don't have the issue.

### Which operating system are you running Bazel on?

Debian GNU/Linux rodete, Linux 5.15.15-1rodete2-amd64, x86-64

### What is the output of `bazel info release`?

INFO: Options provided by the client: Inherited 'common' options: --isatty=1 --terminal_columns=90 INFO: Reading rc options for 'info' from /usr/local/google/home/yeounoh/git/pytorch/xla/third_party/tensorflow/.bazelrc: Inherited 'common' options: --experimental_repo_remote_exec INFO: Reading rc options for 'info' from /usr/local/google/home/yeounoh/git/pytorch/xla/third_party/tensorflow/.bazelrc: Inherited 'build' options: --define framework_shared_object=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library INFO: Reading rc options for 'info' from /usr/local/google/home/yeounoh/git/pytorch/xla/third_party/tensorflow/.tf_configure.bazelrc: Inherited 'build' options: --action_env PYTHON_BIN_PATH=/usr/local/google/home/yeounoh/anaconda3/envs/torch-xla-1.11/bin/python3 --action_env PYTHON_LIB_PATH=/usr/local/google/home/yeounoh/anaconda3/envs/torch-xla-1.11/lib/python3.8/site-packages --python_path=/usr/local/google/home/yeounoh/anaconda3/envs/torch-xla-1.11/bin/python3 INFO: Reading rc options for 'info' from /usr/local/google/home/yeounoh/git/pytorch/xla/third_party/tensorflow/.bazelrc: Inherited 'build' options: --deleted_packages=tensorflow/compiler/mlir/tfrt,tensorflow/compiler/mlir/tfrt/benchmarks,tensorflow/compiler/mlir/tfrt/jit/python_binding,tensorflow/compiler/mlir/tfrt/jit/transforms,tensorflow/compiler/mlir/tfrt/python_tests,tensorflow/compiler/mlir/tfrt/tests,tensorflow/compiler/mlir/tfrt/tests/ir,tensorflow/compiler/mlir/tfrt/tests/analysis,tensorflow/compiler/mlir/tfrt/tests/jit,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_tfrt,tensorflow/compiler/mlir/tfrt/tests/tf_to_corert,tensorflow/compiler/mlir/tfrt/tests/tf_to_tfrt_data,tensorflow/compiler/mlir/tfrt/tests/saved_model,tensorflow/compiler/mlir/tfrt/transforms/lhlo_gpu_to_tfrt_gpu,tensorflow/core/runtime_fallback,tensorflow/core/runtime_fallback/conversion,tensorflow/core/runtime_fallback/kernel,tensorflow/core/runtime_fallback/opdefs,tensorflow/core/runtime_fallback/runtime,tensorflow/core/runtime_fallback/util,tensorflow/core/tfrt/common,tensorflow/core/tfrt/eager,tensorflow/core/tfrt/eager/backends/cpu,tensorflow/core/tfrt/eager/backends/gpu,tensorflow/core/tfrt/eager/core_runtime,tensorflow/core/tfrt/eager/cpp_tests/core_runtime,tensorflow/core/tfrt/gpu,tensorflow/core/tfrt/run_handler_thread_pool,tensorflow/core/tfrt/runtime,tensorflow/core/tfrt/saved_model,tensorflow/core/tfrt/graph_executor,tensorflow/core/tfrt/saved_model/tests,tensorflow/core/tfrt/tpu,tensorflow/core/tfrt/utils INFO: Found applicable config definition build:short_logs in file /usr/local/google/home/yeounoh/git/pytorch/xla/third_party/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING INFO: Found applicable config definition build:v2 in file /usr/local/google/home/yeounoh/git/pytorch/xla/third_party/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1 INFO: Found applicable config definition build:linux in file /usr/local/google/home/yeounoh/git/pytorch/xla/third_party/tensorflow/.bazelrc: --copt=-w --host_copt=-w --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++14 --host_cxxopt=-std=c++14 --config=dynamic_kernels --distinct_host_configuration=false --experimental_guard_against_concurrent_changes INFO: Found applicable config definition build:dynamic_kernels in file /usr/local/google/home/yeounoh/git/pytorch/xla/third_party/tensorflow/.bazelrc: --define=dynamic_loaded_kernels=true --copt=-DAUTOLOAD_DYNAMIC_KERNELS release 5.1.1

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

_No response_

### What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?

```text
https://github.com/tensorflow/tensorflow.git
cb8193c5d2dd82b0a1ecaf78d37392cae8e05582
2c6d3ed00f16838831aa460c5668a8466b9f3649
```

### Have you found anything relevant by searching the web?

_No response_

### Any other information, logs, or outputs that you want to share?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.