abiosoft / abiosoft/colima

image pull times out in 2 minutes (kubernetes k3s)

Abierto
#713 8 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Go
Estrellas
30.8k
Forks
613
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

### Description

Hi,

i am experiencing an issue which seems to be related to k3s config. When pulling through k3s I experience a 2 minute timeout when pulling big images that require more than 2 minutes to be pulled.

An extract of the k8s events follows, I generalized the image name and some details.
```yaml
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 6m24s default-scheduler Successfully assigned some-namespace/pod-name to colima
Warning Failed 2m12s (x2 over 4m25s) kubelet Failed to pull image "registry.corp.org/some-image": rpc error: code = Unknown desc = context deadline exceeded
Warning Failed 2m12s (x2 over 4m25s) kubelet Error: ErrImagePull
Normal BackOff 117s (x2 over 4m24s) kubelet Back-off pulling image "registry.corp.org/some-image"
Warning Failed 117s (x2 over 4m24s) kubelet Error: ImagePullBackOff
Normal Pulling 103s (x3 over 6m24s) kubelet Pulling image "registry.corp.org/some-image"
```

This seems to be related to a default config of k3s. There is an open issue at the "rancher-desktop" repo that seems to point to a possible solution. Here the original issue https://github.com/rancher-sandbox/rancher-desktop/issues/2303

In short it seems that k3s supports a config parameter `"--kubelet-arg=runtime-request-timeout=10m0s"` to e.g. configure this timeout higher.

Unfortunately I do not find an `override.yaml` as described in the rancher-desktop issue, otherwise I would gladly test the fix. I am open to try some fixes if you can provide some high level instructions to follow.

### Version

Colima Version:
```
colima version 0.5.4
git commit: feef4176f56a7dea487d43689317a9d7fe9de27e

runtime: docker
arch: aarch64
client: v23.0.2
server: v20.10.20

kubernetes
Client Version: v1.27.1
Kustomize Version: v5.0.1
Server Version: v1.25.4+k3s1
```

Lima Version: limactl version 0.15.0

Qemu Version: qemu-img version 7.2.0

### Operating System

- [ ] macOS Intel <= 12 (Monterrey)
- [ ] macOS Intel >= 13 (Ventura)
- [ ] macOS M1 <= 12 (Monterrey)
- [X] macOS M1 >= 13 (Ventura)
- [ ] Linux

### Output of `colima status`

INFO[0000] colima is running using QEMU
INFO[0000] arch: aarch64
INFO[0000] runtime: docker
INFO[0000] mountType: sshfs
INFO[0000] socket: unix:///Users/psimona/.colima/default/docker.sock
INFO[0000] kubernetes: enabled

### Reproduction Steps

1. setup colima with kubernetes
2. Ensure that the image is not present on the host with `docker images`, check for the example image `sagemathinc/cocalc`. If present remove with `docker rmi`
4. schedule a pod with a large image (radom example from dockerhub 6gb) to
```
kubectl run big-test-image --image=sagemathinc/cocalc
```
5. Observe deployment events with either via `k9s` (cli interface) or via `kubectl describe pod big-test-image`.
6. After 2 minutes the image pulling will fail with the error message described above `Failed to pull image "sagemathinc/cocalc": rpc error: code = Unknown desc = context deadline exceeded`

### Expected behaviour

Large images can be pulled via colima with kubernetes event if they take longer than 2 minutes to pull.

Ideally this is a sane default and it can be simply configured either per CLI interface or per `colima.yaml`

### Additional context

_No response_

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.