testcontainers / testcontainers/testcontainers-java

LogMessageWaitStrategy misses the required log lines when multiple containers are started

Aperta
#3,186 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Lingua principale
Java
Stelle
8.7k
Fork
1.9k
Merge medio
2g 17h
PR unite (30g)
9

Descrizione

I start a custom database container and then I run a second container which executes a bunch of SQL scripts to initialize the database (these scripts usually run in less than a second) and exits. For both of these containers I use LogMessageWaitStrategy to check that (a) the database is ready; (b) the initialization scripts finished running and the initialization was successful.

The problem: the second container almost always (but not absolutely every time) does not pass the startup check, though the required log line is printed to the log.

After a while I came up with a synthetic test case which does not depend on our custom images.

The Dockerfile:

FROM alpine
COPY init /
CMD ["/init"]

The init script:

#!/bin/sh

if [ "$HANG" = 'true' ]
then
    echo 'I will start and work hard for a long while.'
else
    echo 'I will perform some quick initialization and quit.'
fi

# Emulate some workload before the initialization finishes.
sleep 0.1

echo "Initialization finished."

# Hang to emulate some workload running after initialization.
[ "$HANG" = 'true' ] && sleep inf || true

After building the image with the tag test I run the following test case:

@Test
public void standaloneTest() {
    new GenericContainer<>("test")
        .withEnv("HANG", "true")
        .waitingFor(new LogMessageWaitStrategy()
                .withRegEx(".*Initialization finished.*"))
        .start();

    new GenericContainer<>("test")
        .waitingFor(new LogMessageWaitStrategy()
                .withRegEx(".*Initialization finished.*"))
        .start();
}

The test hangs on the second start() for a while and then fails with Not ready yet exception, though I see the correct log lines reported by Testcontainers just before the exception (Log output from the failed container).

Few things to consider:

  • a single container using the test image always passes the startup check successfully disregarding how quickly it starts and how quickly it exits;
  • a problem appears only when both containers use LogMessageWaitStrategy;
  • increasing the sleep duration in the script decreases the chances of the issue: sleep 0.3 makes the issue less frequent and sleep 1 removes it almost completely;
  • Thread.sleep(1000) inserted between the start()s seems to be a workaround, but it is not reliable and the issue still reproduces, though rarely.

I use Testcontainers 1.14.3. The environment varies; on different machines (all running different flavors of Linux) the issue has a different chance to fire (it seems that the chances are higher on more powerful and fast hardware).

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Direzione di ricerca

Inizia con LogMessageWaitStrategy e riproduci standaloneTest utilizzando il Dockerfile e lo script di inizializzazione forniti, concentrandoti su due container sequenziali con carichi di lavoro diversi. Traccia il modo in cui ciascuna strategia osserva i log dei container e definisci il completamento come il rilevamento affidabile di “Initialization finished.” da parte di entrambi i controlli di avvio, senza il timeout intermittente.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
docker, java, shell
Ambito
devops, testing-qa
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
42/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.