bazel-contrib / bazel-contrib/rules_go

go_path and generated sources results in conflicts when combining architectures

Open
#4,545 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
1.5k
Forks
760
Avg merge
1d 11h
Merged PRs (30d)
12

Description

I have the use case where I need to ship parts of the source code, so that this part can be built externally.

`go_path` works generally pretty well for this as it allows to collect all dependencies and provides a defined path to build from there.

But to make sure that the shipped code is universal, I have to include the sources for different platforms/architectures (e.g. `foo_darwin.go`, `foo_windows.go`, etc).

This is currently solved by defining several targets as entry points (one for each platform/arch pair we support) and having them as dependencies for `go_path`.

Example

```python
PLATFORMS = [
("windows", "amd64"),
("windows", "386"),
("linux", "amd64"),
("linux", "386"),
("linux", "arm"),
("linux", "arm64"),
("darwin", "amd64"),
("darwin", "arm64"),
]
[
go_binary(
name = "binary_%s_%s" % (goos, goarch),
embed = [":binary_lib"],
goarch = goarch,
goos = goos,
visibility = ["//visibility:public"],
)
for goos, goarch in PLATFORMS
]

go_path(
name = "my_gopath",
deps = [
"//:binary_%s_%s" % (goos, goarch)
for goos, goarch in PLATFORMS
],
)
```

This generally works well, but falls apart when generated source files come into play:

```
Traceback (most recent call last):
File "/external/rules_go+/go/private/tools/path.bzl", line 76, column 32, in _go_path_impl
_add_manifest_entry(manifest_entries, manifest_entry_map, inputs, f, dst)
File "/external/rules_go+/go/private/tools/path.bzl", line 277, column 17, in _add_manifest_entry
fail("{}: references multiple files ({} and {})".format(dst, entry_map[dst], src.path))
Error in fail: /file.gen.go: references multiple files (bazel-out/k8-fastbuild-ST-b9eeda35ed41/bin//file.gen.go and bazel-out/k8-fastbuild-ST-1cb734d4f317/bin//file.gen.go)
```

(where `file.gen.go` is created by a `genrule`, only depending on other files, no tools)

Is there a better way of doing this, or can this be otherwise fixed somehow?

Versions
- rules_go: 0.59.0
- gazelle: 0.47.0

Contributor guide

Open the contributing guide

Research direction

Start with go/private/tools/path.bzl, especially _go_path_impl and _add_manifest_entry, and reproduce the go_path setup with architecture-specific targets and generated file.gen.go sources. Trace why identical destination paths from different bazel-out configurations collide. Done means combining the listed platforms succeeds without conflicting manifest entries while preserving the generated sources.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.