GoogleContainerTools / GoogleContainerTools/skaffold

Long 1h+ running Gcloud Builds: Access Tokens Expire

Open
#9,955 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
15.9k
Forks
1.7k
Avg merge
3d 9h
Merged PRs (30d)
10

Description

I have a very long running container build (some R dependencies take a **long** time). After 1 hour I start getting the error in Skaffold output:

```
build [redacted-image-name] failed: error getting build status: googleapi: Error 401: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
```

This error persisted for ~15 minutes (no more output, just this one log line in red). Followed by Skaffold exiting with a more detailed error:

```
Details:
[
{
"@type": "type.googleapis.com/google.rpc.ErrorInfo",
"metadata": {
"method": "google.devtools.cloudbuild.v1.CloudBuild.GetBuild",
"service": "cloudbuild.googleapis.com"
},
"reason": "ACCESS_TOKEN_TYPE_UNSUPPORTED"
}
]
```

My `skaffold.yaml` looks like:

```yaml
apiVersion: skaffold/v4beta11
kind: Config
metadata:
name: redacted-name

build:
artifacts:
- image: redacted-image-name
context: .
docker:
dockerfile: Dockerfile

googleCloudBuild:
projectId: redacted-project-name
timeout: 14400s # 4 hours

profiles:
- name: dev
manifests:
# ...
deploy:
# ...
```

I ran the build with `skaffold run -p dev`.

I am running skaffold version v2.17.1.

I think what's happening is the auth tokens skaffold is using only last for 1 hour. And after an hour they don't get refreshed internally. If I start another build with`skaffold run -p dev` it starts the build just fine again.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.