GoogleContainerTools / GoogleContainerTools/skaffold

When GCB Build fails, skaffold doesn't cancel in process jobs

Open
#7,395 0 comments 0 reactions 0 assignees View on GitHub
area/build area/cleanup build/gcb kind/friction priority/p1
Dominant language
Go
Stars
15.9k
Forks
1.7k
Avg merge
3d 9h
Merged PRs (30d)
10

Description

### Expected behavior

Skaffold cancels any outstanding GCB jobs when one fails and skaffold exits.

### Actual behavior

Any GCB jobs that have been scheduled by skaffold run to completion, taking up Cloud Build task concurrency quota and spending extra compute.

### Information

When a GCB job fails or skaffold loses contact with the service, skaffold immediately quits. Any lingering jobs that have already been scheduled to GCB will continue running, taking up build task concurrency quota (the more import one in my opinion) and costing compute. Unless you are using `tryImportMissing` with `inputDigest` tag, the resulting artifacts will not be known to skaffold and will be rebuilt on the next build so there isn't much point to continuing the build.

- Skaffold version: v1.37.2

### Steps to reproduce the behavior

Run a simple build job that has multiple GCB containers, a concurrency of 0 or > 1 (parallel build), and a build that will fail before they are all complete.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.