testcontainers / testcontainers/testcontainers-java

Failed to start the elasticsearch testcontainers in one CI docker environment (can not change the CI environment file)

Aperta
#5,384 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

resolution/waiting-for-info
Lingua principale
Java
Stelle
8.7k
Fork
1.9k
Merge medio
2g 17h
PR unite (30g)
9

Descrizione

Hi,
I try to build my project and run some test case in CI environment (constructed by common docker image).
It was always failed and tried to get some image from https://registry-1.docker.io/v2/. But it report 500 time out error.
I test two ways (docker-compose start and code start) . Both are failed, my test can be run due the initiation of ElasticInitializer.java.
Below is my question.

  1. why not it can to connect to local ES environment even I start one elasticsearch image in docker compose period.
  2. I provide the internal address in code, but it still report that it can not find the image in that address and try to get from always to https://registry-1.docker.io/v2/.
  3. If I plan to run my test, I must start one image named ryuk:0.3.3 ?

*Test.java
@Testcontainers
@SpringBootTest
@ActiveProfiles(profiles = {"test"})
@ContextConfiguration(initializers = {ElasticInitializer.class})
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
class ServiceImplTest {
}

My CI environment.
Environment
Client: Docker Engine - Community
Version: 20.10.14
API version: 1.41
Go version: go1.16.15
Git commit: a224086
Built: Thu Mar 24 01:47:57 2022
OS/Arch: linux/amd64
Context: default
Experimental: true

Server: Docker Engine - Community
Engine:
Version: 20.10.14
API version: 1.41 (minimum version 1.12)
Go version: go1.16.15
Git commit: 87a90dc
Built: Thu Mar 24 01:45:46 2022
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.5.11
GitCommit: 3df54a852345ae127d1fa3092b95168e4a88e2f8
runc:
Version: 1.0.3
GitCommit: v1.0.3-0-gf46b6ba
docker-init:
Version: 0.19.0
GitCommit: de40ad0

Way 1 : run the testcontainer image in code.

`public class ElasticInitializer
implements ApplicationContextInitializer {
static
DockerImageName myImage = DockerImageName.parse("internal.address.net/elastic/elasticsearch:7.8.0").asCompatibleSubstituteFor("docker.elastic.co/elasticsearch/elasticsearch");
@ClassRule
public final static ElasticsearchContainer ELASTICSEARCH_CONTAINER = new ElasticsearchContainer(myImage).withImagePullPolicy(PullPolicy.defaultPolicy());

/**
 * @param applicationContext
 */
@Override
public void initialize(ConfigurableApplicationContext applicationContext) {
	ELASTICSEARCH_CONTAINER.start();
	TestPropertyValues.of("spring.elasticsearch.uris=" + ELASTICSEARCH_CONTAINER.getHttpHostAddress())
			.applyTo(applicationContext);
}

}`

`09:46:42 01:46:41.133 [main] DEBUG org.testcontainers.utility.TestcontainersConfiguration - Testcontainers configuration overrides will be loaded from file:/root/.testcontainers.properties
09:46:42 01:46:41.149 [main] WARN org.testcontainers.utility.TestcontainersConfiguration - Attempted to read Testcontainers configuration file at file:/root/.testcontainers.properties but the file was not found. Exception message: FileNotFoundException: /root/.testcontainers.properties (No such file or directory)
09:46:42 01:46:41.153 [main] INFO org.testcontainers.utility.ImageNameSubstitutor - Image name substitution will be performed by: DefaultImageNameSubstitutor (composite of 'ConfigurationFileImageNameSubstitutor' and 'PrefixingImageNameSubstitutor')
09:46:42 01:46:41.165 [main] DEBUG org.testcontainers.utility.PrefixingImageNameSubstitutor - No prefix is configured
09:46:42 01:46:41.165 [main] DEBUG org.testcontainers.utility.ImageNameSubstitutor - Did not find a substitute image for internal.address.net/elasstic/elasticsearch:7.8.0 (using image substitutor: DefaultImageNameSubstitutor (composite of 'ConfigurationFileImageNameSubstitutor' and 'PrefixingImageNameSubstitutor'))
09:46:42 01:46:41.189 [main] WARN org.testcontainers.dockerclient.DockerClientProviderStrategy - Can't instantiate a strategy from org.testcontainers.dockerclient.DockerClientProviderStrategy
09:46:42 java.lang.InstantiationException: null
09:46:42 at java.base/jdk.internal.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(InstantiationExceptionConstructorAccessorImpl.java:48)
09:46:42 at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)

09:46:57 01:46:41.962 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - Looking up auth config for image: testcontainers/ryuk:0.3.3 at registry: index.docker.io
09:46:57 01:46:41.963 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - RegistryAuthLocator has configFile: /root/.docker/config.json (does not exist) and commandPathPrefix:
09:46:57 01:46:41.963 [main] INFO org.testcontainers.utility.RegistryAuthLocator - Failure when attempting to lookup auth config. Please ignore if you don't have images in an authenticated registry. Details: (dockerImageName: testcontainers/ryuk:0.3.3, configFile: /root/.docker/config.json. Falling back to docker-java default behaviour. Exception message: /root/.docker/config.json (No such file or directory)
09:46:57 01:46:41.964 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - No matching Auth Configs - falling back to defaultAuthConfig [null]
09:46:57 01:46:41.964 [main] DEBUG org.testcontainers.dockerclient.AuthDelegatingDockerClientConfig - Effective auth config [null]

09:46:57 01:46:41.979 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 >> "POST /v1.32/images/create?fromImage=testcontainers%2Fryuk%3A0.3.3 HTTP/1.1[\r][\n]"
09:46:57 01:46:41.979 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 >> "accept: application/octet-stream[\r][\n]"
09:46:57 01:46:41.979 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 >> "content-type: application/json[\r][\n]"
09:46:57 01:46:41.979 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 >> "Accept-Encoding: gzip, x-gzip, deflate[\r][\n]"
09:46:57 01:46:41.979 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 >> "Host: 127.0.0.1:2375[\r][\n]"
09:46:57 01:46:41.979 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 >> "Connection: keep-alive[\r][\n]"
09:46:57 01:46:41.979 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 >> "User-Agent: Apache-HttpClient/5.0.3 (Java/11.0.10)[\r][\n]"
09:46:57 01:46:41.979 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 >> "[\r][\n]"
09:46:57 01:46:56.980 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 << "HTTP/1.1 500 Internal Server Error[\r][\n]"
09:46:57 01:46:56.980 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 << "Api-Version: 1.40[\r][\n]"
09:46:57 01:46:56.980 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 << "Content-Type: application/json[\r][\n]"
09:46:57 01:46:56.980 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 << "Docker-Experimental: false[\r][\n]"
09:46:57 01:46:56.980 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 << "Ostype: linux[\r][\n]"
09:46:57 01:46:56.980 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 << "Server: Docker/19.03.5 (linux)[\r][\n]"
09:46:57 01:46:56.981 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 << "Date: Wed, 18 May 2022 01:46:56 GMT[\r][\n]"
09:46:57 01:46:56.981 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 << "Content-Length: 157[\r][\n]"
09:46:57 01:46:56.981 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 << "[\r][\n]"
09:46:57 01:46:56.981 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.wire - http-outgoing-0 << "{"message":"Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"}[\n]"

09:46:57 01:46:56.982 [docker-java-stream--1007472360] DEBUG com.github.dockerjava.zerodep.shaded.org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager - ep-00000004: connection released [route: {}->http://127.0.0.1:2375][total/ available: 1; route allocated: 1 of 2147483647; total allocated: 1 of 2147483647]
09:46:57 01:46:56.983 [docker-java-stream--1007472360] ERROR com.github.dockerjava.api.async.ResultCallbackTemplate - Error during callback
09:46:57 com.github.dockerjava.api.exception.InternalServerErrorException: Status 500: {"message":"Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"}
09:46:57
09:46:57 at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.execute(DefaultInvocationBuilder.java:247)
09:46:57 at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.lambda$executeAndStream$1(DefaultInvocationBuilder.java:269)
09:46:57 at java.base/java.lang.Thread.run(Thread.java:834)
`

Way 2
`public class ElasticInitializer
implements ApplicationContextInitializer {
static
DockerImageName myImage = DockerImageName.parse("internal.address.net/elastic/elasticsearch:7.8.0").asCompatibleSubstituteFor("docker.elastic.co/elasticsearch/elasticsearch");
public final static ElasticsearchContainer ELASTICSEARCH_CONTAINER = new ElasticsearchContainer(myImage).withImagePullPolicy(PullPolicy.defaultPolicy());

/**
 * @param applicationContext
 */
@Override
public void initialize(ConfigurableApplicationContext applicationContext) {
	ELASTICSEARCH_CONTAINER.start();

	TestPropertyValues.of("spring.elasticsearch.uris=https://localhost:9200")
		.applyTo(applicationContext);
}

}`

docker-compose.yml
elasticsearch: image: internal.net/elastic/elasticsearch:7.8.0 hostname: elasticsearch ports: - "9200:9200" environment: ELASTIC_PASSWORD: password xpack.security.enabled: "false" xpack.security.http.ssl.enabled: "false" discovery.type: single-node healthcheck: test: curl -f http://127.0.0.1:9200/_cluster/health interval: 30s timeout: 20s retries: 10 networks: - noobaa

pom.xml

    `<docker.host>tcp://127.0.0.1:2375</docker.host>
    <docker.client.strategy>org.testcontainers.dockerclient.EnvironmentAndSystemPropertyClientProviderStrategy</docker.client.strategy>

  <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>testcontainers-bom</artifactId>
            <version>1.16.3</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
           <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>${maven-surefire-plugin.version}</version>
                <configuration>
                    <environmentVariables>
                        <DOCKER_CLIENT_STRATEGY>${docker.client.strategy}</DOCKER_CLIENT_STRATEGY>
                        <DOCKER_HOST>${docker.host}</DOCKER_HOST>
                    </environmentVariables>
                </configuration>
            </plugin>`

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Direzione di ricerca

Inizia con ElasticInitializer.java e ServiceImplTest, quindi confronta la loro configurazione Testcontainers con docker-compose.yml e le proprietà Docker in pom.xml. Riproduci il timeout utilizzando la configurazione Docker registrata e determina se il comportamento segnalato di Elasticsearch locale e Ryuk rappresenta una configurazione supportata; documenta la causa e la risoluzione confermate.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
docker, elasticsearch, java
Ambito
databases, devops, testing-qa
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Da chiarire
Idoneità per principianti
25/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.