GoogleContainerTools / GoogleContainerTools/skaffold

skaffold dev does not rebuild image when using kind with registry

Open
#9,345 6 comments 4 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
15.9k
Forks
1.7k
Avg merge
3d 9h
Merged PRs (30d)
10

Description

### Expected behavior

When running `skaffold dev` using a kind cluster with local registry enabled, editing the Dockerfile should cause the image to rebuild and the container should be re-deployed with the updated image.

### Actual behavior

The image is not rebuilt.

### Information

- Skaffold version: `v2.10.0`
- Operating system: MacOS Sonoma 14.3
- Installed via: Homebrew
- Contents of skaffold.yaml: Using the [getting started](https://github.com/GoogleContainerTools/skaffold/tree/main/examples/getting-started) example

### Steps to reproduce the behavior

1. In this repo, cd `examples/getting-started`
2. Create a `kind` cluster with registry using this script: https://kind.sigs.k8s.io/docs/user/local-registry/
3. `skaffold dev -v info`
4. Edit the Dockerfile, for example add this line to the bottom: `RUN touch test.txt`
5. View the logs; the docker image does not rebuild.
6. To confirm the `test.txt` was not created in the container: `kubectl exec -it getting-started -- ls`
7. To clean up the kind cluster and registry, run `kind delete clusters "kind"; docker network disconnect --force kind kind-registry; docker container rm --force --volumes kind-registry`

Logs from `skaffold dev -v info`:
```
INFO[0001] Skaffold &{Version:v2.10.0 ConfigVersion:skaffold/v4beta9 GitVersion: GitCommit:cbc665bfc1fe7253df466e70dd48e3851d935a3e BuildDate:2024-01-09T08:59:34Z GoVersion:go1.21.0 Compiler:gc Platform:darwin/arm64 User:} subtask=-1 task=DevLoop
INFO[0001] Loaded Skaffold defaults from "/Users/anna.cunningham/.skaffold/config" subtask=-1 task=DevLoop
INFO[0001] map entry found when executing locate for &{skaffold-example . {0x140009fe160 } [] {[] []} [] } of type *latest.Artifact and pointer: 1374399343456 subtask=-1 task=DevLoop
INFO[0001] Using kubectl context: kind-kind subtask=-1 task=DevLoop
INFO[0001] using default-repo=localhost:5001 from cluster configmap subtask=-1 task=DevLoop
INFO[0001] DOCKER_HOST env is not set, using the host from docker context. subtask=-1 task=DevLoop
INFO[0001] no kpt renderer or deployer found, skipping hydrated-dir creation subtask=-1 task=DevLoop
INFO[0001] build concurrency first set to 1 parsed from *runner.pipelineBuilderWithHooks[0] subtask=-1 task=DevLoop
INFO[0001] final build concurrency value is 1 subtask=-1 task=DevLoop
Generating tags...
- skaffold-example -> localhost:5001/skaffold-example:v2.10.0-23-geeaf00144
INFO[0001] Tags generated in 93.908666ms subtask=-1 task=Build
Checking cache...
- skaffold-example: Not found. Building
INFO[0003] Cache check completed in 1.985 second subtask=-1 task=Build
Starting build...
Found [kind-kind] context, using local docker daemon.
Building [skaffold-example]...
Target platforms: [linux/arm64]
#0 building with "orbstack" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 510B done
#1 DONE 0.0s

#2 [internal] load metadata for docker.io/library/golang:1.18
#2 DONE 0.8s

#3 [internal] load metadata for docker.io/library/alpine:3
#3 DONE 0.8s

#4 [internal] load .dockerignore
#4 transferring context: 2B done
#4 DONE 0.0s

#5 [builder 1/5] FROM docker.io/library/golang:1.18@sha256:50c889275d26f816b5314fc99f55425fa76b18fcaf16af255f5d57f09e1f48da
#5 DONE 0.0s

#6 [stage-1 1/2] FROM docker.io/library/alpine:3@sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
#6 DONE 0.0s

#7 [internal] load build context
#7 transferring context: 54B done
#7 DONE 0.0s

#8 [builder 2/5] WORKDIR /code
#8 CACHED

#9 [builder 4/5] COPY go.mod .
#9 CACHED

#10 [builder 5/5] RUN go build -gcflags="${SKAFFOLD_GO_GCFLAGS}" -trimpath -o /app main.go
#10 CACHED

#11 [builder 3/5] COPY main.go .
#11 CACHED

#12 [stage-1 2/2] COPY --from=builder /app .
#12 CACHED

#13 exporting to image
#13 exporting layers done
#13 writing image sha256:eb4f7b79eb739540a81d000ed323c739b22d90435d8b314e833db1c8c7876ef7 done
#13 naming to localhost:5001/skaffold-example:v2.10.0-23-geeaf00144 done
#13 DONE 0.0s
The push refers to repository [localhost:5001/skaffold-example]
445ce1b3ac91: Preparing
b09314aec293: Preparing
445ce1b3ac91: Layer already exists
b09314aec293: Layer already exists
v2.10.0-23-geeaf00144: digest: sha256:1a77ed36ef1f81282f51e6f8162231d7cea094a74ee0bd98633c1008cf2c8def size: 738
Build [skaffold-example] succeeded
INFO[0005] Build completed in 1.575 second subtask=-1 task=Build
INFO[0005] Starting render... subtask=-1 task=DevLoop
INFO[0005] starting render process subtask=0 task=Render
INFO[0005] Render completed in 2.1945ms subtask=-1 task=DevLoop
Tags used in deployment:
- skaffold-example -> localhost:5001/skaffold-example:v2.10.0-23-geeaf00144@sha256:1a77ed36ef1f81282f51e6f8162231d7cea094a74ee0bd98633c1008cf2c8def
Starting deploy...
- pod/getting-started created
Waiting for deployments to stabilize...
- pods is ready.
Deployments stabilized in 3.102 seconds
INFO[0008] Deploy completed in 3.318 seconds subtask=-1 task=Deploy
Listing files to watch...
- skaffold-example
INFO[0008] List generated in 101.125µs subtask=-1 task=DevLoop
Press Ctrl+C to exit
Watching for changes...
INFO[0008] Streaming logs from pod: getting-started container: getting-started subtask=-1 task=DevLoop
[getting-started] Hello world!
[getting-started] Hello world!
```

then after editing the Dockerfile:
```
INFO[0022] files modified: [Dockerfile] subtask=-1 task=DevLoop
Generating tags...
- skaffold-example -> localhost:5001/skaffold-example:v2.10.0-23-geeaf00144-dirty
INFO[0022] Tags generated in 88.819792ms subtask=-1 task=Build
Checking cache...
- skaffold-example: Found Remotely
INFO[0022] Cache check completed in 29.823333ms subtask=-1 task=Build
INFO[0022] Starting render... subtask=-1 task=DevLoop
INFO[0022] starting render process subtask=0 task=Render
INFO[0022] Render completed in 3.210583ms subtask=-1 task=DevLoop
Tags used in deployment:
- skaffold-example -> localhost:5001/skaffold-example:v2.10.0-23-geeaf00144-dirty@sha256:8e32f5ea7a6f02e102714554c1b7fc73f8b0968452bdb89744ac7c9a2610fd59
Starting deploy...
- pod/getting-started configured
Waiting for deployments to stabilize...
INFO[0023] Streaming logs from pod: getting-started container: getting-started subtask=-1 task=DevLoop
- pods is ready.
Deployments stabilized in 2.11 seconds
INFO[0024] Deploy completed in 2.323 seconds subtask=-1 task=Deploy
Watching for changes...
[getting-started] Hello world!
```

On version **2.9.0**, editing the Dockerfile results in these logs; I would expect `2.10.*` to have the same behavior:
```
INFO[0015] files modified: [Dockerfile] subtask=-1 task=DevLoop
Generating tags...
- skaffold-example -> localhost:5001/skaffold-example:v2.10.0-23-geeaf00144-dirty
INFO[0015] Tags generated in 85.997584ms subtask=-1 task=Build
Checking cache...
- skaffold-example: Not found. Building
INFO[0015] Cache check completed in 1.612458ms subtask=-1 task=Build
Starting build...
Found [kind-kind] context, using local docker daemon.
Building [skaffold-example]...
Target platforms: [linux/arm64]
#0 building with "orbstack" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 530B done
#1 DONE 0.0s

#2 [internal] load metadata for docker.io/library/alpine:3
#2 DONE 0.4s

#3 [internal] load metadata for docker.io/library/golang:1.18
#3 DONE 0.4s

#4 [internal] load .dockerignore
#4 transferring context: 2B done
#4 DONE 0.0s

#5 [builder 1/5] FROM docker.io/library/golang:1.18@sha256:50c889275d26f816b5314fc99f55425fa76b18fcaf16af255f5d57f09e1f48da
#5 DONE 0.0s

#6 [stage-1 1/3] FROM docker.io/library/alpine:3@sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b
#6 DONE 0.0s

#7 [internal] load build context
#7 transferring context: 54B done
#7 DONE 0.0s

#8 [builder 2/5] WORKDIR /code
#8 CACHED

#9 [builder 4/5] COPY go.mod .
#9 CACHED

#10 [builder 3/5] COPY main.go .
#10 CACHED

#11 [builder 5/5] RUN go build -gcflags="${SKAFFOLD_GO_GCFLAGS}" -trimpath -o /app main.go
#11 CACHED

#12 [stage-1 2/3] COPY --from=builder /app .
#12 CACHED

#13 [stage-1 3/3] RUN touch test4.txt
#13 DONE 0.1s

#14 exporting to image
#14 exporting layers done
#14 writing image sha256:5efd379ca49ef09490648c2edbdbc3a556687492a3987bd586e39d5092a05e29 done
#14 naming to localhost:5001/skaffold-example:v2.10.0-23-geeaf00144-dirty done
#14 DONE 0.0s
The push refers to repository [localhost:5001/skaffold-example]
69f140f181ce: Preparing
445ce1b3ac91: Preparing
b09314aec293: Preparing
445ce1b3ac91: Layer already exists
b09314aec293: Layer already exists
69f140f181ce: Pushed
v2.10.0-23-geeaf00144-dirty: digest: sha256:39697e8f6a3cf265ccc23fd0b230b97c2cbfcd7011566237707dfcac3a6f5bc9 size: 945
Build [skaffold-example] succeeded
INFO[0016] Build completed in 973.0615ms subtask=-1 task=Build
INFO[0016] Starting render... subtask=-1 task=DevLoop
INFO[0016] starting render process subtask=0 task=Render
INFO[0016] Render completed in 1.511ms subtask=-1 task=DevLoop
Tags used in deployment:
- skaffold-example -> localhost:5001/skaffold-example:v2.10.0-23-geeaf00144-dirty@sha256:39697e8f6a3cf265ccc23fd0b230b97c2cbfcd7011566237707dfcac3a6f5bc9
Starting deploy...
- pod/getting-started configured
Waiting for deployments to stabilize...
INFO[0017] Streaming logs from pod: getting-started container: getting-started subtask=-1 task=DevLoop
- pods is ready.
Deployments stabilized in 2.111 seconds
INFO[0018] Deploy completed in 2.267 seconds subtask=-1 task=Deploy
Watching for changes...
[getting-started] Hello world!
[getting-started] Hello world!
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.