aspect-build / aspect-build/rules_lint

[Bug]: Seeing differences on local vs CI runs for the formatters

Open
#353 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Starlark
Stars
154
Forks
125
Avg merge
3d 21h
Merged PRs (30d)
20

Description

### What happened?

As the title states, starting to pull my hair our locally. Currently I have my format targets configured like..

```starlark
load("@aspect_rules_lint//format:defs.bzl", "format_multirun", "format_test")

format_multirun(
name = "format",
go = "@multitool//tools/gofumpt",
protocol_buffer = "@rules_buf_toolchains//:buf",
starlark = "@buildifier_prebuilt//:buildifier",
terraform = "@multitool//tools/terraform",
yaml = "@multitool//tools/yamlfmt",
)

format_test(
name = "test",
size = "small",
go = "@multitool//tools/gofumpt",
no_sandbox = True,
protocol_buffer = "@rules_buf_toolchains//:buf",
starlark = "@buildifier_prebuilt//:buildifier",
workspace = "//:.gitlab-ci.yml",
yaml = "@multitool//tools/yamlfmt",
)
```

Locally on ubuntu 22.04 and on MacOS a current commit fails as such:
```shell
spencer@gspencer-mlt autobots % bazel test tools/format/... --test_output=summary
INFO: Analyzed 16 targets (0 packages loaded, 0 targets configured).
FAIL: //tools/format:test_Starlark_with_buildifier (see /private/var/tmp/_bazel_gspencer/26b182f6d79bbfc06c198e7527c526e0/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/tools/format/test_Starlark_with_buildifier/test.log)
FAIL: //tools/format:test_YAML_with_yamlfmt (see /private/var/tmp/_bazel_gspencer/26b182f6d79bbfc06c198e7527c526e0/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/tools/format/test_YAML_with_yamlfmt/test.log)
INFO: Found 12 targets and 4 test targets...
INFO: Elapsed time: 0.312s, Critical Path: 0.22s
INFO: 5 processes: 8 local.
INFO: Build completed, 2 tests FAILED, 5 total actions
//tools/format:test_Go_with_gofmt PASSED in 0.1s
//tools/format:test_Protocol_Buffer_with_buf PASSED in 0.1s
//tools/format:test_Starlark_with_buildifier FAILED in 0.1s
/private/var/tmp/_bazel_gspencer/26b182f6d79bbfc06c198e7527c526e0/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/tools/format/test_Starlark_with_buildifier/test.log
//tools/format:test_YAML_with_yamlfmt FAILED in 0.1s
/private/var/tmp/_bazel_gspencer/26b182f6d79bbfc06c198e7527c526e0/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/tools/format/test_YAML_with_yamlfmt/test.log

Executed 4 out of 4 tests: 2 tests pass and 2 fail locally.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
```
Summary here as it is just formatting yaml files and a BUILD file. *

In CI this isn't the case.. as all targets pass
```$ bazel test //tools/format/...
Computing main repo mapping:
Loading:
Loading: 0 packages loaded
Analyzing: 16 targets (1 packages loaded, 0 targets configured)
Analyzing: 16 targets (1 packages loaded, 0 targets configured)
[0 / 1] [Prepa] BazelWorkspaceStatusAction stable-status.txt
Analyzing: 16 targets (16 packages loaded, 1217 targets configured)
[1 / 1] checking cached actions
Analyzing: 16 targets (21 packages loaded, 1257 targets configured)
[1 / 17] checking cached actions
Analyzing: 16 targets (23 packages loaded, 3545 targets configured)
[64 / 69] 4 / 4 tests; [Prepa] Creating symlink bazel-out/k8-fastbuild/bin/external/rules_multitool~~multitool~multitool/tools/terraform/terraform ... (2 actions, 0 running)
INFO: Analyzed 16 targets (23 packages loaded, 3545 targets configured).
INFO: Found 12 targets and 4 test targets...
INFO: Elapsed time: 5.028s, Critical Path: 0.74s
INFO: 76 processes: 68 internal, 8 local.
INFO: Build completed successfully, 76 total actions
//tools/format:test_Go_with_gofmt PASSED in 0.3s
//tools/format:test_Protocol_Buffer_with_buf PASSED in 0.1s
//tools/format:test_Starlark_with_buildifier PASSED in 0.5s
//tools/format:test_YAML_with_yamlfmt PASSED in 0.1s
Executed 4 out of 4 tests: 4 tests pass.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
```

CI here is running in gitlab inside of the `gcr.io/bazel-public/bazel:7.1.0` container image.

I'm not really sure of how to go about debugging this..

### Version

bazel version
Bazelisk version: development
Build label: 7.1.1
Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Build time: Thu Mar 21 18:08:59 2024 (1711044539)
Build timestamp: 1711044539
Build timestamp as int: 1711044539

bazel run @buildifier_prebuilt//:buildifier -- -version
buildifier version: 6.4.0
buildifier scm revision: 433ea8554e82832e4fa4bdd530ca471564071511

bazel run @multitool//tools/yamlfmt -- -version
yamlfmt 0.13.0 (dd8547d)

### How to reproduce

_No response_

### Any other information?

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with the format_multirun and format_test targets shown in the issue, then run bazel test tools/format/... and inspect the test logs for the Starlark and YAML failures. Compare the local Ubuntu/macOS runs with the gcr.io/bazel-public/bazel:7.1.0 CI environment and the listed buildifier and yamlfmt versions. Done means the cross-environment difference is explained and the formatter tests behave consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
yaml
Domain
build-system, testing, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.