"pull QPS exceeded" error shouldn't be fatal
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 10.1k
- Forks
- 413
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 6
Description
Expected Behavior
When a pod fails to pull its image with a "pull QPS exceeded" ErrImagePull error, it might be better to let k8s retry the pull instead of failing. k8s doesn't seem to properly distinguish between permanent and temporary ErrImagePull errors, so I suspect Tilt just fails the job assuming the failure is fatal.
Current Behavior
Failed to pull image "registry.gha-prod.svc.cluster.local:5000/solana-contract:tilt-d0187147de2228a4": pull QPS exceeded
Context
tilt doctor Output
Tilt: v0.25.3, built 2022-03-04
System: linux-amd64
---
Docker
- Host: unix:///docker/docker.sock
- Server Version: 20.10.15
- API Version: 1.41
- Builder: 2
---
Kubernetes
- Env: unknown
- Context: ci
- Cluster Name: Unknown
- Namespace: gha-prod
- Container Runtime: read-failure
- Version: v1.21.2+k3s1
- Cluster Local Registry: {Host:registry.gha-prod.svc.cluster.local:5000 hostFromCluster: SingleName:}
---
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing how Tilt handles Kubernetes ErrImagePull failures, especially the reported "pull QPS exceeded" case. Verify whether temporary image-pull failures can be retried without treating permanent failures as successful, and confirm the behavior against the Kubernetes 1.21.2+k3s1 context described here.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes
- Domain
- infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100