aspect-build / aspect-build/rules_lint
[Bug]: Seeing differences on local vs CI runs for the formatters
- Dominant language
- Starlark
- Stars
- 154
- Forks
- 125
- Avg merge
- 3d 21h
- Merged PRs (30d)
- 20
Description
### What happened?
As the title states, starting to pull my hair our locally. Currently I have my format targets configured like..
```starlark
load("@aspect_rules_lint//format:defs.bzl", "format_multirun", "format_test")
format_multirun(
name = "format",
go = "@multitool//tools/gofumpt",
protocol_buffer = "@rules_buf_toolchains//:buf",
starlark = "@buildifier_prebuilt//:buildifier",
terraform = "@multitool//tools/terraform",
yaml = "@multitool//tools/yamlfmt",
)
format_test(
name = "test",
size = "small",
go = "@multitool//tools/gofumpt",
no_sandbox = True,
protocol_buffer = "@rules_buf_toolchains//:buf",
starlark = "@buildifier_prebuilt//:buildifier",
workspace = "//:.gitlab-ci.yml",
yaml = "@multitool//tools/yamlfmt",
)
```
Locally on ubuntu 22.04 and on MacOS a current commit fails as such:
```shell
spencer@gspencer-mlt autobots % bazel test tools/format/... --test_output=summary
INFO: Analyzed 16 targets (0 packages loaded, 0 targets configured).
FAIL: //tools/format:test_Starlark_with_buildifier (see /private/var/tmp/_bazel_gspencer/26b182f6d79bbfc06c198e7527c526e0/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/tools/format/test_Starlark_with_buildifier/test.log)
FAIL: //tools/format:test_YAML_with_yamlfmt (see /private/var/tmp/_bazel_gspencer/26b182f6d79bbfc06c198e7527c526e0/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/tools/format/test_YAML_with_yamlfmt/test.log)
INFO: Found 12 targets and 4 test targets...
INFO: Elapsed time: 0.312s, Critical Path: 0.22s
INFO: 5 processes: 8 local.
INFO: Build completed, 2 tests FAILED, 5 total actions
//tools/format:test_Go_with_gofmt PASSED in 0.1s
//tools/format:test_Protocol_Buffer_with_buf PASSED in 0.1s
//tools/format:test_Starlark_with_buildifier FAILED in 0.1s
/private/var/tmp/_bazel_gspencer/26b182f6d79bbfc06c198e7527c526e0/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/tools/format/test_Starlark_with_buildifier/test.log
//tools/format:test_YAML_with_yamlfmt FAILED in 0.1s
/private/var/tmp/_bazel_gspencer/26b182f6d79bbfc06c198e7527c526e0/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/tools/format/test_YAML_with_yamlfmt/test.log
Executed 4 out of 4 tests: 2 tests pass and 2 fail locally.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
```
Summary here as it is just formatting yaml files and a BUILD file. *
In CI this isn't the case.. as all targets pass
```$ bazel test //tools/format/...
Computing main repo mapping:
Loading:
Loading: 0 packages loaded
Analyzing: 16 targets (1 packages loaded, 0 targets configured)
Analyzing: 16 targets (1 packages loaded, 0 targets configured)
[0 / 1] [Prepa] BazelWorkspaceStatusAction stable-status.txt
Analyzing: 16 targets (16 packages loaded, 1217 targets configured)
[1 / 1] checking cached actions
Analyzing: 16 targets (21 packages loaded, 1257 targets configured)
[1 / 17] checking cached actions
Analyzing: 16 targets (23 packages loaded, 3545 targets configured)
[64 / 69] 4 / 4 tests; [Prepa] Creating symlink bazel-out/k8-fastbuild/bin/external/rules_multitool~~multitool~multitool/tools/terraform/terraform ... (2 actions, 0 running)
INFO: Analyzed 16 targets (23 packages loaded, 3545 targets configured).
INFO: Found 12 targets and 4 test targets...
INFO: Elapsed time: 5.028s, Critical Path: 0.74s
INFO: 76 processes: 68 internal, 8 local.
INFO: Build completed successfully, 76 total actions
//tools/format:test_Go_with_gofmt PASSED in 0.3s
//tools/format:test_Protocol_Buffer_with_buf PASSED in 0.1s
//tools/format:test_Starlark_with_buildifier PASSED in 0.5s
//tools/format:test_YAML_with_yamlfmt PASSED in 0.1s
Executed 4 out of 4 tests: 4 tests pass.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
```
CI here is running in gitlab inside of the `gcr.io/bazel-public/bazel:7.1.0` container image.
I'm not really sure of how to go about debugging this..
### Version
bazel version
Bazelisk version: development
Build label: 7.1.1
Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Build time: Thu Mar 21 18:08:59 2024 (1711044539)
Build timestamp: 1711044539
Build timestamp as int: 1711044539
bazel run @buildifier_prebuilt//:buildifier -- -version
buildifier version: 6.4.0
buildifier scm revision: 433ea8554e82832e4fa4bdd530ca471564071511
bazel run @multitool//tools/yamlfmt -- -version
yamlfmt 0.13.0 (dd8547d)
### How to reproduce
_No response_
### Any other information?
_No response_
Contributor guide
Research direction
Start with the format_multirun and format_test targets shown in the issue, then run bazel test tools/format/... and inspect the test logs for the Starlark and YAML failures. Compare the local Ubuntu/macOS runs with the gcr.io/bazel-public/bazel:7.1.0 CI environment and the listed buildifier and yamlfmt versions. Done means the cross-environment difference is explained and the formatter tests behave consistently.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- yaml
- Domain
- build-system, testing, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100