testcontainers / testcontainers/testcontainers-java

[Bug]: `withReuse(true)` has a check-then-act race: concurrent starts with the same config create duplicate containers

Offen
#11,979 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Dieses Issue hat noch niemand übernommen.

type/bug
Vorherrschende Sprache
Java
Sterne
8.7k
Forks
1.9k
Ø Merge
2 T. 17 Std.
Gemergte PRs (30 T.)
9

Beschreibung

Module

Core

Testcontainers version

2.0.5

Using the latest Testcontainers version?

Yes

Host OS

macOS (Darwin arm64). Reproduced on this host. The race is in Java-level check-then-act logic with no OS-specific dependency, so it should reproduce on Linux CI hosts as well.

Host Arch

arm64

Docker version
Client:
 Version:           29.2.0
 API version:       1.53
 OS/Arch:           darwin/arm64
Server: Docker Desktop 4.60.1
 Engine:
  Version:          29.2.0
  API version:      1.53 (minimum version 1.44)
  OS/Arch:           linux/arm64
What happened?

GenericContainer.tryStart() implements container reuse (withReuse(true)) as a plain check-then-act sequence, with no synchronization between the check and the act:

String hash = hash(createCommand);
containerId = findContainerForReuse(hash).orElse(null);   // check
if (containerId != null) {
    reused = true;
} else {
    createCommand.getLabels().put(HASH_LABEL, hash);
}
...
if (!reused) {
    containerId = createCommand.exec().getId();            // act
}

findContainerForReuse(String hash) itself still carries a // TODO locking comment:

@VisibleForTesting
Optional<String> findContainerForReuse(String hash) {
    // TODO locking
    return dockerClient.listContainersCmd()...
}

Expected: with withReuse(true), starting the same configuration concurrently should still result in exactly one running container, with the later call(s) reusing the first.

Actual: when two calls to start() with an identical configuration (identical hash) happen concurrently, both can observe "not found" before either finishes creating a container, and both proceed to create one. No exception is thrown either way. Both start() calls return successfully, each attached to its own container. The failure is invisible unless something notices duplicate resource usage.

This happens both across separate JVM processes (the intended cross-run reuse scenario, e.g. separate CI workers or separate Gradle/Maven forks reusing the same container) and within a single JVM across parallel test threads (e.g. JUnit parallel execution sharing a static container field).

Relevant log output

Reproduced with a mock-based test, then again against a real Docker daemon with no artificial delay, using standalone driver programs that call GenericContainer directly (compiled against the module's classpath, launched as separate OS processes/threads, TESTCONTAINERS_REUSE_ENABLE=true).

Control: sequential, no race. Two separate JVM processes, one after another, same config:

seqA containerId=a7c72118...  tookMs=7954  (새로 생성)
seqB containerId=a7c72118...  tookMs=2582  (재사용 성공, 로그: "Reusing container with ID: a7c72118... and hash: eae86aec...")

With no race, reuse works exactly as documented. The defect only shows up under concurrency.

Cross-process race, no artificial delay. Two separate OS processes launched concurrently, each trial using a fresh config (so a fresh hash) to rule out interference between trials, 5 repetitions:

Trial Process A container ID Process B container ID Result
1 111939ed... d1f1f358... duplicate
2 b34c32b2... 3e1ab91b... duplicate
3 d223dde7... 1de0b015... duplicate
4 17ab5cd5... 095eccebc... duplicate
5 f1966cf4... bb47eb0f... duplicate

5/5 (100%) produced duplicate containers. Confirmed with docker inspect directly (bypassing testcontainers) that both containers in each trial shared the identical org.testcontainers.hash label (e.g. trial 1: both 96b1e6d7a323cd9f0a7561d61dcd3924f9135981), so they were meant to be the same reusable resource.

Same-JVM multi-thread race (not cross-process, just multiple threads in one process): 2 threads racing on identical new config both created their own container; with 5 threads, all 5 created distinct containers (0/5 reused).

All containers created during this reproduction were removed afterward (docker rm -f), verified zero leftover.

Additional Information

Related but distinct: #11854 tracks stabilizing the reuse feature toward GA and lists networking, cleanup, and CI-compatibility gaps. It doesn't mention this specific concurrency defect; filing it separately since it's a distinct, concrete bug rather than part of that broader stabilization scope, though it's clearly relevant context for why reuse is still marked experimental.

Fix prepared: a same-JVM lock for in-process races, plus Docker's own container-name uniqueness as a cross-process arbiter for the create call. Please see PR #11980 for details.

Beitragsleitfaden

Beitragsleitfaden öffnen

Erste Schritte

  1. Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
  3. Forke das Repository und arbeite in einem Branch.
  4. Öffne einen Pull Request, der die Issue-Nummer nennt.

Rechercherichtung

Read GenericContainer.tryStart() and findContainerForReuse(), including the TODO locking comment, then review PR #11980 for the prepared fix. Confirm that concurrent starts with identical reuse configuration produce one running container and that later starts reuse it across threads and processes.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
docker, java
Bereich
devops, testing
Issue-Typ
Bug
Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Aktivitätsstatus
Veraltet
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
25/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.