image pull times out in 2 minutes (kubernetes k3s)
- Langage dominant
- Go
- Étoiles
- 30.8k
- Forks
- 613
- Métriques de merge des PR
- Aucune PR mergée en 30 j
Description
### Description
Hi,
i am experiencing an issue which seems to be related to k3s config. When pulling through k3s I experience a 2 minute timeout when pulling big images that require more than 2 minutes to be pulled.
An extract of the k8s events follows, I generalized the image name and some details.
```yaml
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 6m24s default-scheduler Successfully assigned some-namespace/pod-name to colima
Warning Failed 2m12s (x2 over 4m25s) kubelet Failed to pull image "registry.corp.org/some-image": rpc error: code = Unknown desc = context deadline exceeded
Warning Failed 2m12s (x2 over 4m25s) kubelet Error: ErrImagePull
Normal BackOff 117s (x2 over 4m24s) kubelet Back-off pulling image "registry.corp.org/some-image"
Warning Failed 117s (x2 over 4m24s) kubelet Error: ImagePullBackOff
Normal Pulling 103s (x3 over 6m24s) kubelet Pulling image "registry.corp.org/some-image"
```
This seems to be related to a default config of k3s. There is an open issue at the "rancher-desktop" repo that seems to point to a possible solution. Here the original issue https://github.com/rancher-sandbox/rancher-desktop/issues/2303
In short it seems that k3s supports a config parameter `"--kubelet-arg=runtime-request-timeout=10m0s"` to e.g. configure this timeout higher.
Unfortunately I do not find an `override.yaml` as described in the rancher-desktop issue, otherwise I would gladly test the fix. I am open to try some fixes if you can provide some high level instructions to follow.
### Version
Colima Version:
```
colima version 0.5.4
git commit: feef4176f56a7dea487d43689317a9d7fe9de27e
runtime: docker
arch: aarch64
client: v23.0.2
server: v20.10.20
kubernetes
Client Version: v1.27.1
Kustomize Version: v5.0.1
Server Version: v1.25.4+k3s1
```
Lima Version: limactl version 0.15.0
Qemu Version: qemu-img version 7.2.0
### Operating System
- [ ] macOS Intel <= 12 (Monterrey)
- [ ] macOS Intel >= 13 (Ventura)
- [ ] macOS M1 <= 12 (Monterrey)
- [X] macOS M1 >= 13 (Ventura)
- [ ] Linux
### Output of `colima status`
INFO[0000] colima is running using QEMU
INFO[0000] arch: aarch64
INFO[0000] runtime: docker
INFO[0000] mountType: sshfs
INFO[0000] socket: unix:///Users/psimona/.colima/default/docker.sock
INFO[0000] kubernetes: enabled
### Reproduction Steps
1. setup colima with kubernetes
2. Ensure that the image is not present on the host with `docker images`, check for the example image `sagemathinc/cocalc`. If present remove with `docker rmi`
4. schedule a pod with a large image (radom example from dockerhub 6gb) to
```
kubectl run big-test-image --image=sagemathinc/cocalc
```
5. Observe deployment events with either via `k9s` (cli interface) or via `kubectl describe pod big-test-image`.
6. After 2 minutes the image pulling will fail with the error message described above `Failed to pull image "sagemathinc/cocalc": rpc error: code = Unknown desc = context deadline exceeded`
### Expected behaviour
Large images can be pulled via colima with kubernetes event if they take longer than 2 minutes to pull.
Ideally this is a sane default and it can be simply configured either per CLI interface or per `colima.yaml`
### Additional context
_No response_
Guide de contribution
Aucun guide de contribution indexé pour ce dépôt
Évaluation
Cette issue n'a pas encore été évaluée.