testcontainers / testcontainers/testcontainers-java

[Bug]: `withReuse(true)` has a check-then-act race: concurrent starts with the same config create duplicate containers

Aperta
#11,979 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

type/bug
Lingua principale
Java
Stelle
8.7k
Fork
1.9k
Merge medio
2g 17h
PR unite (30g)
9

Descrizione

Module

Core

Testcontainers version

2.0.5

Using the latest Testcontainers version?

Yes

Host OS

macOS (Darwin arm64). Reproduced on this host. The race is in Java-level check-then-act logic with no OS-specific dependency, so it should reproduce on Linux CI hosts as well.

Host Arch

arm64

Docker version
Client:
 Version:           29.2.0
 API version:       1.53
 OS/Arch:           darwin/arm64
Server: Docker Desktop 4.60.1
 Engine:
  Version:          29.2.0
  API version:      1.53 (minimum version 1.44)
  OS/Arch:           linux/arm64
What happened?

GenericContainer.tryStart() implements container reuse (withReuse(true)) as a plain check-then-act sequence, with no synchronization between the check and the act:

String hash = hash(createCommand);
containerId = findContainerForReuse(hash).orElse(null);   // check
if (containerId != null) {
    reused = true;
} else {
    createCommand.getLabels().put(HASH_LABEL, hash);
}
...
if (!reused) {
    containerId = createCommand.exec().getId();            // act
}

findContainerForReuse(String hash) itself still carries a // TODO locking comment:

@VisibleForTesting
Optional<String> findContainerForReuse(String hash) {
    // TODO locking
    return dockerClient.listContainersCmd()...
}

Expected: with withReuse(true), starting the same configuration concurrently should still result in exactly one running container, with the later call(s) reusing the first.

Actual: when two calls to start() with an identical configuration (identical hash) happen concurrently, both can observe "not found" before either finishes creating a container, and both proceed to create one. No exception is thrown either way. Both start() calls return successfully, each attached to its own container. The failure is invisible unless something notices duplicate resource usage.

This happens both across separate JVM processes (the intended cross-run reuse scenario, e.g. separate CI workers or separate Gradle/Maven forks reusing the same container) and within a single JVM across parallel test threads (e.g. JUnit parallel execution sharing a static container field).

Relevant log output

Reproduced with a mock-based test, then again against a real Docker daemon with no artificial delay, using standalone driver programs that call GenericContainer directly (compiled against the module's classpath, launched as separate OS processes/threads, TESTCONTAINERS_REUSE_ENABLE=true).

Control: sequential, no race. Two separate JVM processes, one after another, same config:

seqA containerId=a7c72118...  tookMs=7954  (새로 생성)
seqB containerId=a7c72118...  tookMs=2582  (재사용 성공, 로그: "Reusing container with ID: a7c72118... and hash: eae86aec...")

With no race, reuse works exactly as documented. The defect only shows up under concurrency.

Cross-process race, no artificial delay. Two separate OS processes launched concurrently, each trial using a fresh config (so a fresh hash) to rule out interference between trials, 5 repetitions:

Trial Process A container ID Process B container ID Result
1 111939ed... d1f1f358... duplicate
2 b34c32b2... 3e1ab91b... duplicate
3 d223dde7... 1de0b015... duplicate
4 17ab5cd5... 095eccebc... duplicate
5 f1966cf4... bb47eb0f... duplicate

5/5 (100%) produced duplicate containers. Confirmed with docker inspect directly (bypassing testcontainers) that both containers in each trial shared the identical org.testcontainers.hash label (e.g. trial 1: both 96b1e6d7a323cd9f0a7561d61dcd3924f9135981), so they were meant to be the same reusable resource.

Same-JVM multi-thread race (not cross-process, just multiple threads in one process): 2 threads racing on identical new config both created their own container; with 5 threads, all 5 created distinct containers (0/5 reused).

All containers created during this reproduction were removed afterward (docker rm -f), verified zero leftover.

Additional Information

Related but distinct: #11854 tracks stabilizing the reuse feature toward GA and lists networking, cleanup, and CI-compatibility gaps. It doesn't mention this specific concurrency defect; filing it separately since it's a distinct, concrete bug rather than part of that broader stabilization scope, though it's clearly relevant context for why reuse is still marked experimental.

Fix prepared: a same-JVM lock for in-process races, plus Docker's own container-name uniqueness as a cross-process arbiter for the create call. Please see PR #11980 for details.

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Direzione di ricerca

Read GenericContainer.tryStart() and findContainerForReuse(), including the TODO locking comment, then review PR #11980 for the prepared fix. Confirm that concurrent starts with identical reuse configuration produce one running container and that later starts reuse it across threads and processes.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
docker, java
Ambito
devops, testing
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
25/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.