testcontainers / testcontainers/testcontainers-java

Socat failes after first testclass because it don't waits for docker compose service to be healthy

Open
#2,981 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/docker-compose resolution/acknowledged type/bug
Dominant language
Java
Stars
8.7k
Forks
1.9k
Avg merge
2d 17h
Merged PRs (30d)
9

Description

I have a application which I want to run selenium tests (JUnit 4). I start the application it's database and a elastic search service with a docker compose file. I also start a chrome or firefox browser container depending on a setting. For my first test class this works perfectly but after that the compose services getting restarted. This isn't a problem but it results in socat trying to connect to the application itself right direct after the application container begone to start without waiting for the service to be healthy. Because the application needs a couple of minutes to start the exposed port isn't available and socat fails like this:

ERROR 🐳 [alpine/socat:latest]:485 - Could not start container
org.testcontainers.containers.ContainerLaunchException: Aborting attempt to link to container 1h6gja65n7j4_anwendung_1 as it is not running
	at org.testcontainers.containers.GenericContainer.applyConfiguration(GenericContainer.java:777) ~[testcontainers-1.14.3.jar:?]
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:359) ~[testcontainers-1.14.3.jar:?]
	at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:325) ~[testcontainers-1.14.3.jar:?]
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81) [duct-tape-1.0.8.jar:?]
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:323) [testcontainers-1.14.3.jar:?]
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:311) [testcontainers-1.14.3.jar:?]
	at org.testcontainers.containers.DockerComposeContainer.startAmbassadorContainers(DockerComposeContainer.java:298) [testcontainers-1.14.3.jar:?]
	at org.testcontainers.containers.DockerComposeContainer.start(DockerComposeContainer.java:172) [testcontainers-1.14.3.jar:?]
	at org.testcontainers.containers.DockerComposeContainer.starting(DockerComposeContainer.java:141) [testcontainers-1.14.3.jar:?]
	at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:29) [testcontainers-1.14.3.jar:?]
	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.jar:4.13]
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) [junit-4.13.jar:4.13]
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413) [junit-4.13.jar:4.13]
	at org.junit.runners.Suite.runChild(Suite.java:128) [junit-4.13.jar:4.13]
	at org.junit.runners.Suite.runChild(Suite.java:27) [junit-4.13.jar:4.13]
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) [junit-4.13.jar:4.13]
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) [junit-4.13.jar:4.13]
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) [junit-4.13.jar:4.13]
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) [junit-4.13.jar:4.13]
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) [junit-4.13.jar:4.13]
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) [junit-4.13.jar:4.13]
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413) [junit-4.13.jar:4.13]
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) [surefire-junit4-2.22.2.jar:2.22.2]
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) [surefire-junit4-2.22.2.jar:2.22.2]
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) [surefire-junit4-2.22.2.jar:2.22.2]
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) [surefire-junit4-2.22.2.jar:2.22.2]
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384) [surefire-booter-2.22.2.jar:2.22.2]
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345) [surefire-booter-2.22.2.jar:2.22.2]
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) [surefire-booter-2.22.2.jar:2.22.2]
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418) [surefire-booter-2.22.2.jar:2.22.2]

Since I am not allowed to post the whole code, here is an excerpt with the most important:

@ClassRule
    public static final DockerComposeContainer environment = new DockerComposeContainer(
            new File("src/test/resources/compose-test.yml"))
            .withPull(true)
            .withEnv("CONF_PATH", System.getProperty("CONF_PATH"))
            .withEnv("CONF_FOLDER", System.getProperty("CONF_FOLDER"))
            .withEnv("DOCKER_TAG", System.getProperty("DOCKER_TAG")).withLocalCompose(true)
            .waitingFor(SERVICE_ANWENDUNG, Wait.forHealthcheck().withStartupTimeout(Duration.ofMinutes(60)))
            .withExposedService(SERVICE_ANWENDUNG, SERVICE_ANWENDUNG_PORT,
                    Wait.forHealthcheck().withStartupTimeout(Duration.ofMinutes(60)));
@Rule
    public BrowserWebDriverContainer browser = configureBrowserContainer();

         //In a other class:
public BrowserWebDriverContainer configureBrowserContainer() {
        if (browserContainer == null) {
            browserContainer = new BrowserWebDriverContainer<>()
                    .withImagePullPolicy(PullPolicy.alwaysPull())
                    .withCapabilities(getBrowserCapabilities())
                    .withNetwork(Network.SHARED)
                    .withNetworkAliases("vnchost")
                    .withEnv("SCREEN_WIDTH", "1920")
                    .withEnv("SCREEN_HEIGHT", "1080")
                    .waitingFor(Wait.defaultWaitStrategy().withStartupTimeout(Duration.ofMinutes(20)))
                    .withClasspathResourceMapping("test_data/", "/test_data", BindMode.READ_ONLY)
                    .withRecordingMode(BrowserWebDriverContainer.VncRecordingMode.RECORD_FAILING, new File("results"));
        }
        return browserContainer;
    }

My docker compose file looks like this:

version: '2.2'
services:
  elasticsearch:
    image: my.repo.org/myappl/elasticsearch:7.4.2
  oracle:
    image: my.repo.org/myappl/oracle_test:oracle_19_3_0
  myappl:
    build:
      context: ${CONF_PATH}
      args:
        - VERSION=${DOCKER_TAG}
        - CONF_FOLDER=${CONF_FOLDER}
    depends_on:
      elasticsearch:
        condition: service_healthy
      oracle:
        condition: service_healthy
    expose:
      - 8080
    dns_search:
      - some.dns.org
    links:
      - elasticsearch
      - oracle
    cap_add:
      - DAC_READ_SEARCH

And this are the dependencies I am using:

<properties>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
        <selenium-driver.version>3.141.59</selenium-driver.version>
</properties>

<dependencies>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>1.7.30</version>
            <scope>test</scope>
        </dependency>
        <!-- Binding for Log4J -->
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-slf4j-impl</artifactId>
            <version>2.12.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-api</artifactId>
            <version>2.13.3</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
            <version>2.13.3</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-configuration2</artifactId>
            <version>2.7</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-remote-driver</artifactId>
            <version>${selenium-driver.version}</version>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-firefox-driver</artifactId>
            <version>${selenium-driver.version}</version>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-edge-driver</artifactId>
            <version>${selenium-driver.version}</version>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-ie-driver</artifactId>
            <version>${selenium-driver.version}</version>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>${selenium-driver.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
            <version>3.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
            <version>4.5.12</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-support</artifactId>
            <version>4.0.0-alpha-5</version>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.13</version>
        </dependency>
        <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>selenium</artifactId>
            <version>1.14.3</version>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>junit</groupId>
                    <artifactId>junit</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.22.2</version>
                <configuration>
                    <includes>
                        <include>**/${testsuite}</include>
                    </includes>
                    <systemPropertyVariables>
                        <CONF_PATH>${project.basedir}/conf/</CONF_PATH>
                        <CONF_FOLDER>myappl</CONF_FOLDER>
                        <BROWSER_TO_USE>${selenium.browser}</BROWSER_TO_USE>
                        <DOCKER_TAG>latest-nightly</DOCKER_TAG>
                    </systemPropertyVariables>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-enforcer-plugin</artifactId>
                <version>3.0.0-M3</version>
                <executions>
                    <execution>
                        <id>enforce-maven</id>
                        <goals>
                            <goal>enforce</goal>
                        </goals>
                        <configuration>
                            <rules>
                                <requireMavenVersion>
                                    <version>3.6.3</version>
                                </requireMavenVersion>
                            </rules>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

    <profiles>
        <profile>
            <id>chrome</id>
            <properties>
                <selenium.browser>chrome</selenium.browser>
            </properties>
        </profile>
        <profile>
            <id>firefox</id>
            <properties>
                <selenium.browser>firefox</selenium.browser>
            </properties>
        </profile>
        <profile>
            <id>all</id>
            <properties>
                <testsuite>*TestSuite.java</testsuite>
            </properties>
        </profile>
        <profile>
            <id>BasicTestSuite</id>
            <properties>
                <testsuite>BasicTestSuite.java</testsuite>
            </properties>
        </profile>
    </profiles>

I run it with ./mvnw clean test -Pchrome,BasicTestSuite

My docker version:

Client: Docker Engine - Community
 Version:           19.03.12
 API version:       1.40
 Go version:        go1.13.10
 Git commit:        48a66213fe
 Built:             Mon Jun 22 15:45:44 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.12
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.10
  Git commit:       48a66213fe
  Built:            Mon Jun 22 15:44:15 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

And docker compose:

docker-compose version 1.25.0, build unknown
docker-py version: 4.1.0
CPython version: 3.8.2
OpenSSL version: OpenSSL 1.1.1f  31 Mar 2020

Java version:

openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)

I have managed to reproduce the problem in a sample project: https://github.com/Nicklas2751/testcontainers-issue-example
To run it and reproduce the problem: ./mvnw clean test -Pchrome,BasicTestSuite

Let me know if you need more information or if I can help.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The relevant entry point named by the trace is DockerComposeContainer.startAmbassadorContainers; compare its startup sequence with the withExposedService(... Wait.forHealthcheck()) setup in the JUnit 4 class and compose-test.yml. Reproduce through the Maven Surefire test configuration and confirm that restarting the compose services does not make socat start before the application is healthy.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, docker-compose, java
Domain
devops, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.