GoogleContainerTools / GoogleContainerTools/skaffold
Build concurrency with bazel builds
- Dominant language
- Go
- Stars
- 15.9k
- Forks
- 1.7k
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 10
Description
Am interested in guidance on how best to use skaffold to run our CI builds. We're moving to using bazel for our artifact builds, and I was interested in how concurrency would be handled by the bazel server. It appears skaffold invokes multiple bazel builds, and bazel server only allows these to be run one at a time. Eg if I set `concurrency: 4` and run skaffold build with 4 bazel artifacts, I get
```
Building 4 artifacts in parallel
Building [blah]...
Another command holds the client lock:
pid=35039
owner=client
cwd=/Users/robgates/Repositories/blah
Waiting for it to complete...
Another command holds the client lock:
pid=35026
owner=client
cwd=/Users/robgates/Repositories/blah
Waiting for it to complete...
Another command (pid=35039) is running. Waiting for it to complete on the server (server_pid=10136)..
```
... bazel waits until each build completes before allowing the next to start.
Is there a better way of setting this up? Is there any way for skaffold to invoke `bazel build //src/artifact1:artifact1.tar //src/artifact2:artifact2.tar //src/artifact3:artifact3.tar //src/artifact4:artifact4.tar` such that bazel handles the concurrency? (obviously I know with multiple providers they'll all behave differently so I understand the complexity; but am just wondering what the right way to handle this scenario is?
Contributor guide
Assessment
This issue has not been assessed yet.