GoogleContainerTools / GoogleContainerTools/skaffold

Unclear error messaging for failed pod startup

Open
#4,547 3 comments 0 reactions 0 assignees View on GitHub
area/status-check kind/bug priority/p3
Dominant language
Go
Stars
15.9k
Forks
1.7k
Avg merge
3d 9h
Merged PRs (30d)
10

Description

I'm trying to run triage party with the IntelliJ Cloud Run support (minikube + skaffold).
I know my setup is wrong (no github token, etc.) but I wanted to see the "Cloud run emulation" telling me that - well instead I get

```
Waiting for deployments to stabilize...
- deployment/triage-party: waiting for rollout to finish: 0 of 1 updated replicas are available...
- pod/triage-party-77df66658b-bn42x: creating container triage-party-container
- deployment/triage-party: waiting for rollout to finish: 0 of 1 updated replicas are available...
- pod/triage-party-77df66658b-bn42x: container triage-party-container terminated with exit code 1
Error retrieving logs for pod triage-party-77df66658b-bn42x. Try `kubectl logs triage-party-77df66658b-bn42x -n default -c triage-party-container`
```

repro:

https://github.com/google/triage-party
![image](https://user-images.githubusercontent.com/635073/88305670-9db5a080-ccbe-11ea-85d3-32fb96db82c0.png)

![image](https://user-images.githubusercontent.com/635073/88305709-aa39f900-ccbe-11ea-8a0c-f67c4444846c.png)

---
Full log:

```
Preparing Google Cloud SDK (this may take several minutes for first time setup)...

Starting the Cloud Run development session...

Generating temporary Kubernetes pods and services configuration ...
/Users/balintp/Library/Application Support/google-cloud-tools-java/managed-cloud-sdk/LATEST/google-cloud-sdk/bin/gcloud alpha code export --service-name triage-party --image-name triage-party --no-skaffold-file --kubernetes-file /var/folders/lk/h2pmhk456999cwnrm162297400fv8w/T/cloud-code-temp-k8s2400057970674858815.tmp --env-vars K_SERVICE=triage-party,K_REVISION=local,K_CONFIGURATION=triage-party --memory-limit 256MiB --quiet
WARNING: The --image-name option is deprecated; use --image instead.
WARNING: The --memory-limit option is deprecated; use --memory instead.

Generating a temporary skaffold.yaml configuration...
/Users/balintp/Library/Application Support/cloud-code/bin/versions/8409c589921e86c7502501bcf008deaea7066f73286b757f4c2f083cea56fc28/skaffold init --filename /var/folders/lk/h2pmhk456999cwnrm162297400fv8w/T/cloud-code-temp-skaffold12857831915761732936.tmp --force --artifact {"builder":"Docker","payload":{"path":"Dockerfile"},"image":"triage-party"} --XXenableJibInit --kubernetes-manifest /var/folders/lk/h2pmhk456999cwnrm162297400fv8w/T/cloud-code-temp-k8s2400057970674858815.tmp
Configuration /var/folders/lk/h2pmhk456999cwnrm162297400fv8w/T/cloud-code-temp-skaffold12857831915761732936.tmp was written
You can now run [skaffold build] to build the artifacts
or [skaffold run] to build and deploy
or [skaffold dev] to enter development mode, with auto-redeploy

Configuring sha256 tagging policy in /var/folders/lk/h2pmhk456999cwnrm162297400fv8w/T/cloud-code-temp-skaffold12857831915761732936.tmp...
Configuring image push to false in /var/folders/lk/h2pmhk456999cwnrm162297400fv8w/T/cloud-code-temp-skaffold12857831915761732936.tmp...
Configuring status health check deadline to 300 seconds...

Minikube may take several minutes to start up for the first time...

Checking status of cloud-code-minikube Minikube cluster...
/Users/balintp/Library/Application Support/google-cloud-tools-java/managed-cloud-sdk/LATEST/google-cloud-sdk/bin/minikube status --profile cloud-code-minikube --output json
* There is no local cluster named "cloud-code-minikube"
- To fix this, run: "minikube start -p cloud-code-minikube"

Status of Minikube api-server is UNDEFINED
Starting Minikube with cloud-code-minikube profile...
/Users/balintp/Library/Application Support/google-cloud-tools-java/managed-cloud-sdk/LATEST/google-cloud-sdk/bin/minikube start --profile cloud-code-minikube --keep-context true --wait false --vm-driver docker --interactive false --delete-on-failure
* [cloud-code-minikube] minikube v1.12.0 on Darwin 10.15.5
- MINIKUBE_WANTUPDATENOTIFICATION=false
* Using the docker driver based on user configuration
* Starting control plane node cloud-code-minikube in cluster cloud-code-minikube
* Downloading Kubernetes v1.18.3 preload ...
> preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 2.23 MiB / > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 6.12 MiB / > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 11.86 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 18.44 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 24.87 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 31.50 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 38.17 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 44.95 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 51.50 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 58.22 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 64.91 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 71.56 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 78.19 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 84.88 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 91.55 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 98.41 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 104.95 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 111.64 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 118.44 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 125.17 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 128.20 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 134.84 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 140.77 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 145.23 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 150.64 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 157.45 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 163.97 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 170.69 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 177.41 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 183.95 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 190.38 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 195.88 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 201.95 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 208.61 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 215.03 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 221.66 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 228.36 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 234.92 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 241.70 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 248.23 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 254.83 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 261.34 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 268.08 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 274.52 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 281.14 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 284.23 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 287.85 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 294.14 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 300.79 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 307.48 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 309.64 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 313.09 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 320.04 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 326.65 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 333.23 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 339.92 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 346.57 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 353.04 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 359.81 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 366.10 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 370.90 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 375.82 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 382.56 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 389.32 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 395.90 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 402.68 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 408.29 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 414.31 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 418.90 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 425.53 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 432.32 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 438.95 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 445.74 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 452.37 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 458.98 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 465.71 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 472.60 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 479.24 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 485.98 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 492.49 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 499.31 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 505.95 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 512.74 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 519.46 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 526.06 MiB > preloaded-images-k8s-v4-v1.18.3-docker-overlay2-amd64.tar.lz4: 526.27 MiB* Creating docker container (CPUs=2, Memory=5901MB) ...
* Preparing Kubernetes v1.18.3 on Docker 19.03.2 ...
* Verifying Kubernetes components...
* Enabled addons: default-storageclass, storage-provisioner
* To connect to this cluster, use: kubectl --context=cloud-code-minikube

Minikube started successfully
Starting Cloud Run dev session...
/Users/balintp/Library/Application Support/cloud-code/bin/versions/8409c589921e86c7502501bcf008deaea7066f73286b757f4c2f083cea56fc28/skaffold dev --filename /var/folders/lk/h2pmhk456999cwnrm162297400fv8w/T/cloud-code-temp-skaffold12857831915761732936.tmp --rpc-port 50051 --skip-tests=true --port-forward=true --status-check=true --enable-rpc=true --kube-context cloud-code-minikube --minikube-profile cloud-code-minikube
Listing files to watch...
- triage-party
Generating tags...
- triage-party -> triage-party:latest
Checking cache...
- triage-party: Not found. Building
Found [cloud-code-minikube] context, using local docker daemon.
Building [triage-party]...
Sending build context to Docker daemon 17.24MB
Step 1/24 : FROM golang AS builder
latest: Pulling from library/golang
31dd5ebca5ef: Pulling fs layer
3ed641c4ae98: Pulling fs layer
bcd57146431e: Pulling fs layer
ac34a4d7c330: Pulling fs layer
e3faf675e54f: Pulling fs layer
6dce2b085ae0: Pulling fs layer
3d864a24b2ed: Pulling fs layer
e3faf675e54f: Waiting
6dce2b085ae0: Waiting
3d864a24b2ed: Waiting
ac34a4d7c330: Waiting
3ed641c4ae98: Verifying Checksum
3ed641c4ae98: Download complete
bcd57146431e: Download complete
31dd5ebca5ef: Verifying Checksum
31dd5ebca5ef: Download complete
e3faf675e54f: Verifying Checksum
e3faf675e54f: Download complete
3d864a24b2ed: Verifying Checksum
3d864a24b2ed: Download complete
31dd5ebca5ef: Pull complete
3ed641c4ae98: Pull complete
bcd57146431e: Pull complete
6dce2b085ae0: Download complete
ac34a4d7c330: Verifying Checksum
ac34a4d7c330: Download complete
ac34a4d7c330: Pull complete
e3faf675e54f: Pull complete
6dce2b085ae0: Pull complete
3d864a24b2ed: Pull complete
Digest: sha256:2aa6447d32de9bd2156c5aa379acb2766c196415626d06d8f5fcc2167e6f8844
Status: Downloaded newer image for golang:latest
---> a794da9351a3
Step 2/24 : WORKDIR /app
---> Running in 51b13b3cf812
---> 8310dcff50a3
Step 3/24 : ENV SRC_DIR=/src/tparty
---> Running in d54a977fd3cc
---> b213228f5c31
Step 4/24 : ENV GO111MODULE=on
---> Running in 0c8bb5762c47
---> fe28bddc7f4d
Step 5/24 : RUN mkdir -p ${SRC_DIR}/cmd ${SRC_DIR}/third_party ${SRC_DIR}/pkg ${SRC_DIR}/site /app/third_party /app/site
---> Running in 9d3a81718460
---> 0409ae00fc64
Step 6/24 : COPY go.* $SRC_DIR/
---> d1f0b858e85f
Step 7/24 : COPY cmd ${SRC_DIR}/cmd/
---> 31c2f53a44b6
Step 8/24 : COPY pkg ${SRC_DIR}/pkg/
---> 411e2a48f5b2
Step 9/24 : WORKDIR $SRC_DIR
---> Running in fb64e5b9abec
---> f47c8272f8fb
Step 10/24 : RUN go mod download
---> Running in 5c38b45f581f
---> 6cae578912cd
Step 11/24 : RUN go build cmd/server/main.go
---> Running in ff75473fa8d5
---> a893e62d3a58
Step 12/24 : FROM alpine AS temp
latest: Pulling from library/alpine
df20fa9351a1: Pulling fs layer
df20fa9351a1: Download complete
df20fa9351a1: Pull complete
Digest: sha256:185518070891758909c9f839cf4ca393ee977ac378609f700f60a771a2dfe321
Status: Downloaded newer image for alpine:latest
---> a24bb4013296
Step 13/24 : ARG CFG=config/config.yaml
---> Running in 71b8d8520b53
---> e2408fc7d7c2
Step 14/24 : COPY pcache /pc
---> 0c0e1398bd1b
Step 15/24 : RUN echo "Pre-populating cache if found (failure is perfectly OK)"
---> Running in bcda8c0556d3
Pre-populating cache if found (failure is perfectly OK)
---> 1bccaf428fd6
Step 16/24 : RUN mv /pc/$(basename $CFG).pc /config.yaml.pc || touch /config.yaml.pc
---> Running in 370cf229b35f
mv: can't rename '/pc/config.yaml.pc': No such file or directory
---> d63939ebb0f1
Step 17/24 : FROM gcr.io/distroless/base AS triage-party
latest: Pulling from distroless/base
24f0c933cbef: Pulling fs layer
69e2f037cdb3: Pulling fs layer
24f0c933cbef: Verifying Checksum
24f0c933cbef: Download complete
69e2f037cdb3: Verifying Checksum
69e2f037cdb3: Download complete
24f0c933cbef: Pull complete
69e2f037cdb3: Pull complete
Digest: sha256:2b0a8e9a13dcc168b126778d9e947a7081b4d2ee1ee122830d835f176d0e2a70
Status: Downloaded newer image for gcr.io/distroless/base:latest
---> c4c70323b0c2
Step 18/24 : ARG CFG=config/config.yaml
---> Running in 7ba7915c934a
---> 68ac0235d4cd
Step 19/24 : COPY --from=builder /src/tparty/main /app/
---> c0df6c92b5cb
Step 20/24 : COPY --from=temp /config.yaml.pc /app/pcache/config.yaml.pc
---> 6d43e442fb3d
Step 21/24 : COPY site /app/site/
---> a6e5d6dd840a
Step 22/24 : COPY third_party /app/third_party/
---> c69fb44a2d1a
Step 23/24 : COPY $CFG /app/config/config.yaml
---> 32bc39fa731a
Step 24/24 : CMD ["/app/main", "--min-refresh=30s", "--max-refresh=8m", "--site=/app/site", "--3p=/app/third_party"]
---> Running in df6044e71382
---> bf82dae7f6e5
Successfully built bf82dae7f6e5
Successfully tagged triage-party:latest
Tags used in deployment:
- triage-party -> triage-party:bf82dae7f6e56df5ef2d05c28871d09b8a59e87672c61dd2ce91946abef3b643
Starting deploy...
- deployment.apps/triage-party created
- service/triage-party created
Waiting for deployments to stabilize...
- deployment/triage-party: waiting for rollout to finish: 0 of 1 updated replicas are available...
- pod/triage-party-77df66658b-bn42x: creating container triage-party-container
- deployment/triage-party: waiting for rollout to finish: 0 of 1 updated replicas are available...
- pod/triage-party-77df66658b-bn42x: container triage-party-container terminated with exit code 1
Error retrieving logs for pod triage-party-77df66658b-bn42x. Try `kubectl logs triage-party-77df66658b-bn42x -n default -c triage-party-container`

```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.