GoogleContainerTools / GoogleContainerTools/skaffold

skaffold run inside Google Cloud Build self-cancels

Open
#10,110 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
15.9k
Forks
1.7k
Avg merge
3d 9h
Merged PRs (30d)
10

Description

Title: skaffold run inside Google Cloud Build self-cancels at ~1–2 min (exit 0,
deploy skipped, build reports SUCCESS) since ~2026-06-08 — independent of
skaffold version, builder, and logging mode

## Summary
Since ~2026-06-08, `skaffold run` executed inside a Google Cloud Build step
aborts ~60–130 s into the build phase: it prints "Build [X] was canceled" for
every in-flight artifact, prints its normal version-check/exit banner, and exits
0. Cloud Build therefore records the step as SUCCESS, but the deploy phase never
runs. This pipeline worked unchanged for years before this date.

## Environment
- Invoked as: gcloud builds submit --config=cloudbuild.yaml (from GitHub Actions
and reproduced manually).
- Build step image: gcr.io/k8s-skaffold/skaffold — reproduced on v2.4.0, v2.16.0,
and v2.22.0 (identical behavior).
- Builders tried: googleCloudBuild profile (nested builds) AND default local
docker builder — both die identically.
- options.logging tried: LEGACY and CLOUD_LOGGING_ONLY — both die identically
(the CLOUD_LOGGING_ONLY run streamed no logs to the client and still died).
- skaffold config: apiVersion skaffold/v1beta2 (auto-upgraded to v4beta14),
7 docker artifacts, kubectl deployer. Default pool. Build timeout 5400s.

## Symptom (from `gcloud builds describe`)
- status: SUCCESS, no failureInfo/statusDetail.
- BUILD phase duration ~98–165 s; skaffold has not finished even the first image
when it cancels. timeout (5400s) nowhere near hit.
- Example build IDs (all SUCCESS, ~2 min, no deploy): 62f94e95-c90f-4bbc-bbe4-
ee4593163e4f, a98e43b0-312b-45e8-80db-a913945e882f, 09241dce-0789-42b1-a308-
e3bfa76bc6b8.

## Ruled out
- Not a Cloud Build timeout/cancel/failure (build record = clean SUCCESS).
- Not skaffold version (v2.4 / v2.16 / v2.22).
- Not the builder (gcb nested AND local docker both die).
- Not logging / log-streaming (LEGACY and CLOUD_LOGGING_ONLY both die).
- bash/docker/gcloud/kubectl in the SAME step run fine well past the cancel
(post-skaffold shell commands always execute).

_a trivial sleep 300 step in the same project runs the full 5 min and succeeds, so the Cloud Build environment is healthy; the termination is specific to skaffold_

## Expected vs actual
Expected: skaffold builds all artifacts and runs the kubectl deploy; or if
something cancels it, it exits non-zero so Cloud Build reports FAILURE.
Actual: skaffold cancels all builds ~2 min in and exits 0; deploy skipped; build
is falsely green.

## Question
What terminates the skaffold process ~2 min into a Cloud Build step (only
skaffold reacts; sibling shell processes survive), and why does skaffold exit 0
rather than failing?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.