GoogleContainerTools / GoogleContainerTools/skaffold

GCB builder may exhaust `get` quota with large number of build artifacts

Open
#5,888 0 comments 0 reactions 0 assignees View on GitHub
area/build build/gcb kind/feature-request priority/p2 source/partnerships
Dominant language
Go
Stars
15.9k
Forks
1.7k
Avg merge
3d 9h
Merged PRs (30d)
10

Description

GCB has a quota of 660req/min on `get`s to check build results. When dealing with builds with many artifacts, with a larger team, it's not impossible to hit this threshold.

We currently have [an exponential backoff](https://github.com/GoogleContainerTools/skaffold/blob/ff5038a51e8408e5ac80f66b6867ff2b4ba528c2/pkg/skaffold/build/gcb/types.go#L71-L78) that starts at 1s and increases by 1.5x per step to a maximum of 10 steps with a cap of 60s. That means we could check 8 times per minute per container. With 15 containers in the project, that could be hit if 5 people are building simultaneously (e.g., incl CI builds).

GCB supports a [`list` operation](https://cloud.google.com/build/docs/api/reference/rest/v1/projects.builds/list) that takes a filter. We should be able to use this operation for the suite of builds that have been created and reduce our API use.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.