GoogleContainerTools / GoogleContainerTools/skaffold

Build concurrency with bazel builds

Open
#6,047 1 comment 8 reactions 0 assignees View on GitHub
build/bazel kind/feature-request priority/p3
Dominant language
Go
Stars
15.9k
Forks
1.7k
Avg merge
3d 9h
Merged PRs (30d)
10

Description

Am interested in guidance on how best to use skaffold to run our CI builds. We're moving to using bazel for our artifact builds, and I was interested in how concurrency would be handled by the bazel server. It appears skaffold invokes multiple bazel builds, and bazel server only allows these to be run one at a time. Eg if I set `concurrency: 4` and run skaffold build with 4 bazel artifacts, I get
```
Building 4 artifacts in parallel
Building [blah]...
Another command holds the client lock:
pid=35039
owner=client
cwd=/Users/robgates/Repositories/blah

Waiting for it to complete...
Another command holds the client lock:
pid=35026
owner=client
cwd=/Users/robgates/Repositories/blah

Waiting for it to complete...
Another command (pid=35039) is running. Waiting for it to complete on the server (server_pid=10136)..
```
... bazel waits until each build completes before allowing the next to start.

Is there a better way of setting this up? Is there any way for skaffold to invoke `bazel build //src/artifact1:artifact1.tar //src/artifact2:artifact2.tar //src/artifact3:artifact3.tar //src/artifact4:artifact4.tar` such that bazel handles the concurrency? (obviously I know with multiple providers they'll all behave differently so I understand the complexity; but am just wondering what the right way to handle this scenario is?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.