testcontainers / testcontainers/testcontainers-java

LogMessageWaitStrategy misses the required log lines when multiple containers are started

Offen
#3,186 2 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Dieses Issue hat noch niemand übernommen.

Vorherrschende Sprache
Java
Sterne
8.7k
Forks
1.9k
Ø Merge
2 T. 17 Std.
Gemergte PRs (30 T.)
9

Beschreibung

I start a custom database container and then I run a second container which executes a bunch of SQL scripts to initialize the database (these scripts usually run in less than a second) and exits. For both of these containers I use LogMessageWaitStrategy to check that (a) the database is ready; (b) the initialization scripts finished running and the initialization was successful.

The problem: the second container almost always (but not absolutely every time) does not pass the startup check, though the required log line is printed to the log.

After a while I came up with a synthetic test case which does not depend on our custom images.

The Dockerfile:

FROM alpine
COPY init /
CMD ["/init"]

The init script:

#!/bin/sh

if [ "$HANG" = 'true' ]
then
    echo 'I will start and work hard for a long while.'
else
    echo 'I will perform some quick initialization and quit.'
fi

# Emulate some workload before the initialization finishes.
sleep 0.1

echo "Initialization finished."

# Hang to emulate some workload running after initialization.
[ "$HANG" = 'true' ] && sleep inf || true

After building the image with the tag test I run the following test case:

@Test
public void standaloneTest() {
    new GenericContainer<>("test")
        .withEnv("HANG", "true")
        .waitingFor(new LogMessageWaitStrategy()
                .withRegEx(".*Initialization finished.*"))
        .start();

    new GenericContainer<>("test")
        .waitingFor(new LogMessageWaitStrategy()
                .withRegEx(".*Initialization finished.*"))
        .start();
}

The test hangs on the second start() for a while and then fails with Not ready yet exception, though I see the correct log lines reported by Testcontainers just before the exception (Log output from the failed container).

Few things to consider:

  • a single container using the test image always passes the startup check successfully disregarding how quickly it starts and how quickly it exits;
  • a problem appears only when both containers use LogMessageWaitStrategy;
  • increasing the sleep duration in the script decreases the chances of the issue: sleep 0.3 makes the issue less frequent and sleep 1 removes it almost completely;
  • Thread.sleep(1000) inserted between the start()s seems to be a workaround, but it is not reliable and the issue still reproduces, though rarely.

I use Testcontainers 1.14.3. The environment varies; on different machines (all running different flavors of Linux) the issue has a different chance to fire (it seems that the chances are higher on more powerful and fast hardware).

Beitragsleitfaden

Beitragsleitfaden öffnen

Erste Schritte

  1. Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
  3. Forke das Repository und arbeite in einem Branch.
  4. Öffne einen Pull Request, der die Issue-Nummer nennt.

Rechercherichtung

Beginne mit LogMessageWaitStrategy und reproduziere standaloneTest mithilfe des bereitgestellten Dockerfile und Init-Skripts, wobei der Fokus auf zwei aufeinanderfolgenden Containern mit unterschiedlichen Workloads liegt. Verfolge, wie jede Strategie die Container-Logs beobachtet, und definiere den Abschluss so, dass beide Startup-Prüfungen zuverlässig „Initialization finished.“ erkennen, ohne dass der sporadische Timeout auftritt.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
docker, java, shell
Bereich
devops, testing-qa
Issue-Typ
Bug
Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Aktivitätsstatus
Veraltet
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
42/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.