[Bug]: Deadlock between DockerClientFactory and RyukResourceReaper with JUnit 5 parallel tests

Module

Core

Testcontainers version

1.20.1

Using the latest Testcontainers version?

Yes

Host OS

MacOS

Host Arch

arm64

Docker version

Client:
 Version:           27.1.1
 API version:       1.46
 Go version:        go1.21.12
 Git commit:        6312585
 Built:             Tue Jul 23 19:54:12 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.33.0 (160616)
 Engine:
  Version:          27.1.1
  API version:      1.46 (minimum version 1.24)
  Go version:       go1.21.12
  Git commit:       cc13f95
  Built:            Tue Jul 23 19:57:14 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.7.19
  GitCommit:        2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
 runc:
  Version:          1.7.19
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

What happened?

I'm attempting to run tests in parallel with JUnit 5. One test spins up a static ComposeContainer with .withLocalCompose(true) and another spins up a static KafkaContainer. This leads to a deadlock on startup, where one thread acquires the lock on RyukResourceReaper and then fails to acquire the lock in DockerClientFactory, while the other thread does the opposite.

Relevant log output

Kafka container thread:

"testcontainers-lifecycle-0" #35 [41731] daemon prio=5 os_prio=31 cpu=207.18ms elapsed=23.45s tid=0x000000012225ba00 nid=41731 waiting for monitor entry  [0x00000001735fa000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at org.testcontainers.utility.RyukResourceReaper.maybeStart(RyukResourceReaper.java:74)
	- waiting to lock <0x000000060201f118> (a org.testcontainers.utility.RyukResourceReaper)
	at org.testcontainers.utility.RyukResourceReaper.init(RyukResourceReaper.java:42)
	at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:232)
	- locked <0x000000060201ee60> (a [Ljava.lang.Object;)
	at org.testcontainers.DockerClientFactory$1.getDockerClient(DockerClientFactory.java:106)
	at com.github.dockerjava.api.DockerClientDelegate.authConfig(DockerClientDelegate.java:109)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:329)

Compose container thread:

"testcontainers-lifecycle-1" #37 [37891] daemon prio=5 os_prio=31 cpu=3.14ms elapsed=23.41s tid=0x0000000122254600 nid=37891 waiting for monitor entry  [0x0000000173a12000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:185)
	- waiting to lock <0x000000060201ee60> (a [Ljava.lang.Object;)
	at org.testcontainers.DockerClientFactory$1.getDockerClient(DockerClientFactory.java:106)
	at com.github.dockerjava.api.DockerClientDelegate.authConfig(DockerClientDelegate.java:109)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:329)
	at org.testcontainers.utility.RyukResourceReaper.maybeStart(RyukResourceReaper.java:78)
	- locked <0x000000060201f118> (a org.testcontainers.utility.RyukResourceReaper)
	at org.testcontainers.utility.RyukResourceReaper.registerLabelsFilterForCleanup(RyukResourceReaper.java:51)
	at org.testcontainers.containers.ComposeDelegate.registerContainersForShutdown(ComposeDelegate.java:247)
	at org.testcontainers.containers.ComposeContainer.start(ComposeContainer.java:125)
	- locked <0x000000060201f2a8> (a java.lang.Object)

Additional Information

No response

Start with DockerClientFactory.java and RyukResourceReaper.java, using the reported thread dumps to trace the opposing lock order during parallel startup. Reproduce the JUnit 5 scenario with a static ComposeContainer using local Compose and a static KafkaContainer; done means both containers start without a deadlock.