bazel-contrib / bazel-contrib/rules_go

gopackagesdriver returns an invalid set of files for stdlib in race mode

Open
#4,683 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Go
Stars
1.5k
Forks
760
Avg merge
1d 11h
Merged PRs (30d)
12

Description

Full reproducer at https://github.com/bazel-contrib/rules_go/pull/4682.

As title says, the `gopackagesdriver` returns an invalid set of files for the stdlib in race mod - to understand the issue, we start by looking at `consts.go`: https://cs.opensource.google/go/go/+/refs/tags/go1.24.0:src/internal/runtime/sys/consts.go;l=15

`isRace` is defined in the two sibling files `consts_{norace,race}.go`, each mutually excluded via `race` tag.
My understanding of the workings here is that the `stdlib.pkg.json` gets produced by `go list -tags race ...` at https://github.com/bazel-contrib/rules_go/blob/9fb5d2b23e9a008d4d4d96d7fd210624e98dedd0/go/tools/builders/stdliblist.go#L272-L275
The list returned here should name `internal/runtime/sys/consts_race.go` and omits `consts_norace.go`.

The list/stdlib json is then surfaced via the `go_pkg_info_aspect` and used by the `gopackagesdriver`, but is filtered again at https://github.com/bazel-contrib/rules_go/blob/9fb5d2b23e9a008d4d4d96d7fd210624e98dedd0/go/tools/gopackagesdriver/packageregistry.go#L76

here, it seems that the `build.Context` is unaware of `race` in `GOTAGS`, so ends up filtering away `consts_race.go` https://github.com/bazel-contrib/rules_go/blob/9fb5d2b23e9a008d4d4d96d7fd210624e98dedd0/go/tools/gopackagesdriver/build_context.go#L9-L25

resulting in the `isRace` symbol being undefined since `consts_race.go` is excluded from the `pkg.CompiledGoFiles`.

For a reproducer example, see https://github.com/bazel-contrib/rules_go/pull/4682.

---

I'm not exactly sure what the solution is - for full disclosure, I've been troubleshooting the above with help of AI and it has suggested to not filter the stdlib to prevent this issue, as that list is the only one that arrives already "prefiltered" via `go list` cmd (unlike the `pkg.json` for non-stdlib Go packages in Bazel whose srcs contains all files).

```diff
diff --git go/tools/gopackagesdriver/packageregistry.go go/tools/gopackagesdriver/packageregistry.go
index 4290973f..e8ff2a25 100644
--- go/tools/gopackagesdriver/packageregistry.go
+++ go/tools/gopackagesdriver/packageregistry.go
@@ -71,9 +71,16 @@ func (pr *PackageRegistry) Add(pkgs ...*FlatPackage) *PackageRegistry {
}

func (pr *PackageRegistry) ResolvePaths(prf PathResolverFunc) error {
+ stdlibIDs := make(map[string]struct{}, len(pr.stdlib))
+ for _, pkg := range pr.stdlib {
+ stdlibIDs[pkg.ID] = struct{}{}
+ }
+
for _, pkg := range pr.packagesByID {
ResolvePaths(pkg, prf)
- FilterFilesForBuildTags(pkg)
+ if _, isStdlib := stdlibIDs[pkg.ID]; !isStdlib {
+ FilterFilesForBuildTags(pkg)
+ }
}
return nil
}
```

I've confirmed the above would work for me, however, to me the root of the issue rather seems like the `gopackagesdriver` should be aware of the `race` config (and/or any other build tags) _automatically_ and appriopriately use it for filtering. Possibly this needs to be a separate configuration file dumped by the aspect that can be read back by the `gopackagesdriver`?

What do you think?

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue using the example in pull request 4682. Read stdliblist.go around lines 272-275, then trace filtering in packageregistry.go around line 76 and build_context.go lines 9-25. Done means the race-mode standard library package reports consts_race.go rather than consts_norace.go, without regressing filtering for non-stdlib packages.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
build-system, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.