GoogleContainerTools / GoogleContainerTools/skaffold

Skaffold needs to propagate `minikube docker-env` to builders

Open
#7,156 7 comments 0 reactions 0 assignees View on GitHub
area/build area/env-vars area/local build/jib kind/bug platform/minikube priority/p2
Dominant language
Go
Stars
15.9k
Forks
1.7k
Avg merge
3d 9h
Merged PRs (30d)
10

Description

@ddobrin was attempting to use Skaffold and Jib with a Spring Boot project. This project includes tests that use testcontainers.org to spin up emulators using docker containers from Java code. The tests fail when run from Skaffold as Skaffold isn't propagating the `minikube docker-env` to the Jib build, and so the tests try spinning up containers with Docker Desktop's daemon rather than minikube's daemon.

```
2022-03-03 08:24:58.940 INFO 57968 --- [ main] com.example.QuotesRepositoryTest : Starting QuotesRepositoryTest using Java 11.0.13 on ddobrin-macbookpro.roam.corp.google.com with PID 57968 (started by ddobrin in /Users/ddobrin/work/dan/modern-java-google-cloud/demo-app)
2022-03-03 08:24:58.941 INFO 57968 --- [ main] com.example.QuotesRepositoryTest : The following 1 profile is active: "test"
2022-03-03 08:24:59.833 INFO 57968 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2022-03-03 08:24:59.979 INFO 57968 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 139 ms. Found 1 JPA repository interfaces.
2022-03-03 08:25:00.551 INFO 57968 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
2022-03-03 08:25:00.669 INFO 57968 --- [ main] o.t.d.DockerClientProviderStrategy : Loaded org.testcontainers.dockerclient.UnixSocketClientProviderStrategy from ~/.testcontainers.properties, will try it first
2022-03-03 08:25:01.559 INFO 57968 --- [ main] o.t.d.DockerClientProviderStrategy : Found Docker environment with Environment variables, system properties and defaults. Resolved dockerHost=tcp://127.0.0.1:65029
2022-03-03 08:25:01.559 INFO 57968 --- [ main] org.testcontainers.DockerClientFactory : Docker host IP address is 127.0.0.1
2022-03-03 08:25:01.615 INFO 57968 --- [ main] org.testcontainers.DockerClientFactory : Connected to docker:
Server Version: 20.10.8
API Version: 1.41
Operating System: Ubuntu 20.04.2 LTS
Total Memory: 11979 MB
2022-03-03 08:25:01.618 INFO 57968 --- [ main] o.t.utility.ImageNameSubstitutor : Image name substitution will be performed by: DefaultImageNameSubstitutor (composite of 'ConfigurationFileImageNameSubstitutor' and 'PrefixingImageNameSubstitutor')
2022-03-03 08:25:02.354 WARN 57968 --- [containers-ryuk] o.testcontainers.utility.ResourceReaper : Can not connect to Ryuk at 127.0.0.1:49154

java.net.ConnectException: Connection refused (Connection refused)
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:na]
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399) ~[na:na]
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242) ~[na:na]
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224) ~[na:na]
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:na]
at java.base/java.net.Socket.connect(Socket.java:609) ~[na:na]
at org.testcontainers.utility.ResourceReaper.lambda$null$3(ResourceReaper.java:194) ~[testcontainers-1.16.3.jar:na]
at org.rnorth.ducttape.ratelimits.RateLimiter.doWhenReady(RateLimiter.java:27) ~[duct-tape-1.0.8.jar:na]
at org.testcontainers.utility.ResourceReaper.lambda$start$4(ResourceReaper.java:190) ~[testcontainers-1.16.3.jar:na]
at java.base/java.lang.Thread.run(Thread.java:829) ~[na:na]

```

We should propagate the `minikube docker-env` for all tools being invoked from Skaffold.

There are a few workarounds, but none are ideal as the Cloud Run emulation case uses a separate minikube profile and which may not exist until Cloud Code launches the app with the Cloud Run emulator.
1. Configure surefire's environment to with the values of `minikube docker-env -p ...`
2. Set the environment before launching the IDE: `eval $(minikube docker-env -p ...); code .`

cc: @sujit-kamireddy

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.