GoogleContainerTools / GoogleContainerTools/skaffold

Local Kind cluster hangs on Waiting for deployments to stabilize

Open
#8,898 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
15.9k
Forks
1.7k
Avg merge
3d 9h
Merged PRs (30d)
10

Description

Don't know how to reproduce but it happens quite often, let's say 1 out of 3 cluster start. When it does, it timeouts and I have to destroy/recreate the local Kind cluster for it to work again. I've seen this hang for at least 2 years, but I've never gotten around to creating an issue as I thought that at some point I would update skaffold and it would be fixed.

This never happened on a real cluster, only on a local Kind cluster.

When it hangs the output is like this
```
Waiting for deployments to stabilize...
- myproject-local:deployment/myproject-backend-depl: waiting for deployment spec update to be observed...
- myproject-local:deployment/myproject-editor-frontend-depl: waiting for deployment spec update to be observed...
- myproject-local:deployment/myproject-mailhog: waiting for deployment spec update to be observed...
- myproject-local:statefulset/myproject-mysql: Waiting for statefulset spec update to be observed...
```

Then after recreating the Kind cluster the output is like this and it works.
```
Waiting for deployments to stabilize...
- myproject-local:deployment/myproject-editor-frontend-depl is ready. [3/4 deployment(s) still pending]
- myproject-local:deployment/myproject-backend-depl is ready. [2/4 deployment(s) still pending]
- myproject-local:deployment/myproject-mailhog: creating container mailhog
- myproject-local:pod/myproject-mailhog-649569d68b-s8rsx: creating container mailhog
- myproject-local:statefulset/myproject-mysql: waiting for init container volume-permissions to start
- myproject-local:pod/myproject-mysql-0: waiting for init container volume-permissions to start
- myproject-local:deployment/myproject-mailhog is ready. [1/4 deployment(s) still pending]
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.