testcontainers / testcontainers/testcontainers-java
Failed to start the elasticsearch testcontainers in one CI docker environment (can not change the CI environment file)
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 1.9k
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 9
Description
Hi,
I try to build my project and run some test case in CI environment (constructed by common docker image).
It was always failed and tried to get some image from https://registry-1.docker.io/v2/. But it report 500 time out error.
I test two ways (docker-compose start and code start) . Both are failed, my test can be run due the initiation of ElasticInitializer.java.
Below is my question.
- why not it can to connect to local ES environment even I start one elasticsearch image in docker compose period.
- I provide the internal address in code, but it still report that it can not find the image in that address and try to get from always to https://registry-1.docker.io/v2/.
- If I plan to run my test, I must start one image named ryuk:0.3.3 ?
*Test.java
@Testcontainers
@SpringBootTest
@ActiveProfiles(profiles = {"test"})
@ContextConfiguration(initializers = {ElasticInitializer.class})
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
class ServiceImplTest {
}
My CI environment.
Environment
Client: Docker Engine - Community
Version: 20.10.14
API version: 1.41
Go version: go1.16.15
Git commit: a224086
Built: Thu Mar 24 01:47:57 2022
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.14
API version: 1.41 (minimum version 1.12)
Go version: go1.16.15
Git commit: 87a90dc
Built: Thu Mar 24 01:45:46 2022
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.5.11
GitCommit: 3df54a852345ae127d1fa3092b95168e4a88e2f8
runc:
Version: 1.0.3
GitCommit: v1.0.3-0-gf46b6ba
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Way 1 : run the testcontainer image in code.
`public class ElasticInitializer
implements ApplicationContextInitializer {
static
DockerImageName myImage = DockerImageName.parse("internal.address.net/elastic/elasticsearch:7.8.0").asCompatibleSubstituteFor("docker.elastic.co/elasticsearch/elasticsearch");
@ClassRule
public final static ElasticsearchContainer ELASTICSEARCH_CONTAINER = new ElasticsearchContainer(myImage).withImagePullPolicy(PullPolicy.defaultPolicy());
/**
* @param applicationContext
*/
@Override
public void initialize(ConfigurableApplicationContext applicationContext) {
ELASTICSEARCH_CONTAINER.start();
TestPropertyValues.of("spring.elasticsearch.uris=" + ELASTICSEARCH_CONTAINER.getHttpHostAddress())
.applyTo(applicationContext);
}
}`
`09:46:42 01:46:41.133 [main] DEBUG org.testcontainers.utility.TestcontainersConfiguration - Testcontainers configuration overrides will be loaded from file:/root/.testcontainers.properties
09:46:42 01:46:41.149 [main] WARN org.testcontainers.utility.TestcontainersConfiguration - Attempted to read Testcontainers configuration file at file:/root/.testcontainers.properties but the file was not found. Exception message: FileNotFoundException: /root/.testcontainers.properties (No such file or directory)
09:46:42 01:46:41.153 [main] INFO org.testcontainers.utility.ImageNameSubstitutor - Image name substitution will be performed by: DefaultImageNameSubstitutor (composite of 'ConfigurationFileImageNameSubstitutor' and 'PrefixingImageNameSubstitutor')
09:46:42 01:46:41.165 [main] DEBUG org.testcontainers.utility.PrefixingImageNameSubstitutor - No prefix is configured
09:46:42 01:46:41.165 [main] DEBUG org.testcontainers.utility.ImageNameSubstitutor - Did not find a substitute image for internal.address.net/elasstic/elasticsearch:7.8.0 (using image substitutor: DefaultImageNameSubstitutor (composite of 'ConfigurationFileImageNameSubstitutor' and 'PrefixingImageNameSubstitutor'))
09:46:42 01:46:41.189 [main] WARN org.testcontainers.dockerclient.DockerClientProviderStrategy - Can't instantiate a strategy from org.testcontainers.dockerclient.DockerClientProviderStrategy
09:46:42 java.lang.InstantiationException: null
09:46:42 at java.base/jdk.internal.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(InstantiationExceptionConstructorAccessorImpl.java:48)
09:46:42 at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
09:46:57 01:46:41.962 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - Looking up auth config for image: testcontainers/ryuk:0.3.3 at registry: index.docker.io
09:46:57 01:46:41.963 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - RegistryAuthLocator has configFile: /root/.docker/config.json (does not exist) and commandPathPrefix:
09:46:57 01:46:41.963 [main] INFO org.testcontainers.utility.RegistryAuthLocator - Failure when attempting to lookup auth config. Please ignore if you don't have images in an authenticated registry. Details: (dockerImageName: testcontainers/ryuk:0.3.3, configFile: /root/.docker/config.json. Falling back to docker-java default behaviour. Exception message: /root/.docker/config.json (No such file or directory)
09:46:57 01:46:41.964 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - No matching Auth Configs - falling back to defaultAuthConfig [null]
09:46:57 01:46:41.964 [main] DEBUG org.testcontainers.dockerclient.AuthDelegatingDockerClientConfig - Effective auth config [null]
09:46:57 01:46:41.979 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 >> "POST /v1.32/images/create?fromImage=testcontainers%2Fryuk%3A0.3.3 HTTP/1.1[\r][\n]"
09:46:57 01:46:41.979 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 >> "accept: application/octet-stream[\r][\n]"
09:46:57 01:46:41.979 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 >> "content-type: application/json[\r][\n]"
09:46:57 01:46:41.979 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 >> "Accept-Encoding: gzip, x-gzip, deflate[\r][\n]"
09:46:57 01:46:41.979 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 >> "Host: 127.0.0.1:2375[\r][\n]"
09:46:57 01:46:41.979 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 >> "Connection: keep-alive[\r][\n]"
09:46:57 01:46:41.979 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 >> "User-Agent: Apache-HttpClient/5.0.3 (Java/11.0.10)[\r][\n]"
09:46:57 01:46:41.979 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 >> "[\r][\n]"
09:46:57 01:46:56.980 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 << "HTTP/1.1 500 Internal Server Error[\r][\n]"
09:46:57 01:46:56.980 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 << "Api-Version: 1.40[\r][\n]"
09:46:57 01:46:56.980 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 << "Content-Type: application/json[\r][\n]"
09:46:57 01:46:56.980 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 << "Docker-Experimental: false[\r][\n]"
09:46:57 01:46:56.980 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 << "Ostype: linux[\r][\n]"
09:46:57 01:46:56.980 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 << "Server: Docker/19.03.5 (linux)[\r][\n]"
09:46:57 01:46:56.981 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 << "Date: Wed, 18 May 2022 01:46:56 GMT[\r][\n]"
09:46:57 01:46:56.981 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 << "Content-Length: 157[\r][\n]"
09:46:57 01:46:56.981 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 << "[\r][\n]"
09:46:57 01:46:56.981 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 << "{"message":"Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"}[\n]"
09:46:57 01:46:56.982 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager - ep-00000004: connection released [route: {}->http://127.0.0.1:2375][total/ available: 1; route allocated: 1 of 2147483647; total allocated: 1 of 2147483647]
09:46:57 01:46:56.983 [docker-java-stream--1007472360] ERROR com.github.dockerjava.api.async.ResultCallbackTemplate - Error during callback
09:46:57 com.github.dockerjava.api.exception.InternalServerErrorException: Status 500: {"message":"Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"}
09:46:57
09:46:57 at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.execute(DefaultInvocationBuilder.java:247)
09:46:57 at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.lambda$executeAndStream$1(DefaultInvocationBuilder.java:269)
09:46:57 at java.base/java.lang.Thread.run(Thread.java:834)
`
Way 2
`public class ElasticInitializer
implements ApplicationContextInitializer {
static
DockerImageName myImage = DockerImageName.parse("internal.address.net/elastic/elasticsearch:7.8.0").asCompatibleSubstituteFor("docker.elastic.co/elasticsearch/elasticsearch");
public final static ElasticsearchContainer ELASTICSEARCH_CONTAINER = new ElasticsearchContainer(myImage).withImagePullPolicy(PullPolicy.defaultPolicy());
/**
* @param applicationContext
*/
@Override
public void initialize(ConfigurableApplicationContext applicationContext) {
ELASTICSEARCH_CONTAINER.start();
TestPropertyValues.of("spring.elasticsearch.uris=https://localhost:9200")
.applyTo(applicationContext);
}
}`
docker-compose.yml
elasticsearch:
image: internal.net/elastic/elasticsearch:7.8.0
hostname: elasticsearch
ports:
- "9200:9200"
environment:
ELASTIC_PASSWORD: password
xpack.security.enabled: "false"
xpack.security.http.ssl.enabled: "false"
discovery.type: single-node
healthcheck:
test: curl -f http://127.0.0.1:9200/_cluster/health
interval: 30s
timeout: 20s
retries: 10
networks:
- noobaa
pom.xml
`<docker.host>tcp://127.0.0.1:2375</docker.host>
<docker.client.strategy>org.testcontainers.dockerclient.EnvironmentAndSystemPropertyClientProviderStrategy</docker.client.strategy>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers-bom</artifactId>
<version>1.16.3</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<configuration>
<environmentVariables>
<DOCKER_CLIENT_STRATEGY>${docker.client.strategy}</DOCKER_CLIENT_STRATEGY>
<DOCKER_HOST>${docker.host}</DOCKER_HOST>
</environmentVariables>
</configuration>
</plugin>`
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with ElasticInitializer.java and ServiceImplTest, then compare their Testcontainers setup with docker-compose.yml and the Docker properties in pom.xml. Reproduce the timeout using the logged Docker configuration and determine whether the reported local Elasticsearch and Ryuk behavior represents a supported setup; document the confirmed cause and resolution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, elasticsearch, java
- Domain
- databases, devops, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100