facebook / facebook/buck2

Go rules improvements

Open
#240 6 comments 1 reaction 0 assignees View on GitHub
prelude
Dominant language
Rust
Stars
4.4k
Forks
394
PR merge metrics
No merged PRs in 30d

Description

I want to start making some contributions toward buck2 support for Go, but the existing rules left me puzzled:

1. It's not possible to build a go_binary today. That's because there is no place in the existing rules that would include the go standard library to compile / link packages against. In Bazel's rules_go, stdlib is a stand alone action-artifact that would get included automatically into all compile / link actions.

2. `prelude//go/tools:testmaingen` could not be built with the native prelude tool for the same reason as (1). So testing with go_test is also not possible. This is solved in rules_go with `go_tool_binary`.

Example stacktrace:
```
sluongng/remote-go-toolchain ~/work/facebook/buck2> buck2 build prelude//go/...
File changed: prelude//toolchains/go.bzl
File changed: root//.git/index.lock
File changed: root//.git/index
Action failed: prelude//go/tools:testmaingen (go_compile main)
Local command returned non-zero exit code 2
Reproduce locally: `env "GOARCH=arm64" "GOOS=darwin" "GOROOT=buck-out/v2/gen-anon/anon/213ed1b7ab8693791b34881e73f7aa00/ ...... rwin_arm64/pack" @buck-out/v2/gen/prelude/213ed1b7ab869379/go/tools/__testmaingen__/main.go.argsfile (run `buck2 log what-failed` to get the full command)`
stdout:
prelude/go/tools/testmaingen.go:21:2: could not import bufio (file not found)
prelude/go/tools/testmaingen.go:22:2: could not import bytes (file not found)
prelude/go/tools/testmaingen.go:23:2: could not import errors (file not found)
prelude/go/tools/testmaingen.go:24:2: could not import flag (file not found)
prelude/go/tools/testmaingen.go:25:2: could not import fmt (file not found)
prelude/go/tools/testmaingen.go:26:2: could not import go/ast (file not found)
prelude/go/tools/testmaingen.go:27:2: could not import go/build (file not found)
prelude/go/tools/testmaingen.go:28:2: could not import go/doc (file not found)
prelude/go/tools/testmaingen.go:29:2: could not import go/parser (file not found)
prelude/go/tools/testmaingen.go:30:2: could not import go/scanner (file not found)
prelude/go/tools/testmaingen.go:30:2: too many errors
stderr:
Build ID: 0320ba5b-ee98-4d2c-b773-32188b23ef47
Jobs completed: 19. Time elapsed: 0.1s. Cache hits: 0%. Commands: 2 (cached: 0, remote: 0, local: 2)
BUILD FAILED
Failed to build 'prelude//go/tools:testmaingen (prelude//platforms:default#213ed1b7ab869379)'
exit 3
```

3. It's unclear to me on how to create a language rule set to support multiple toolchains. In Bazel, we could include both the linux and darwin go toolchains in a project: darwin could be used to build locally while linux toolchain is used for remote build. Remote build could be a normal build, or could be a cross build (linux -> darwin, darwin -> linux etc...). I do see in the current Zig toolchain there is a support for `target` platform, but what about `execution` platform? Is there such a concept in Buck2?

4. Trusted release binary. In rules_go, we would use Bazel to request the content from one of these 2 URLS
```
"https://go.dev/dl/?mode=json&include=all",
"https://golang.google.cn/dl/?mode=json&include=all",
```

This download request does not come with a checksum, thus it will not be cached.
They are considered "trusted" source of Go toolchain download and the json outputs include the sha256 checksum for subsequent toolchain downloads.

However, in Buck2, the `actions.download_file` is more restrictive than Bazel's: it only allows you to download with a checksum. Since I could still accomplish the same thing if I were to write a bash/python binary to do all the downloads, I would suggest allowing sha256/sha1 to be optional for `actions.download_file` and when it's not there, simply don't cache the action outputs.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.