testcontainers / testcontainers/testcontainers-java

[Bug]: ResourceReaper.start hangs waiting for ReentrantLock when TLS enabled

Open
#10,454 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type/bug
Dominant language
Java
Stars
8.7k
Forks
1.9k
Avg merge
2d 17h
Merged PRs (30d)
9

Description

Module

Core

Testcontainers version

1.15.2

Using the latest Testcontainers version?

No

Host OS

Ubuntu 22.04.5 LTS

Host Arch

x86_64

Docker version
Client: Docker Engine - Community
 Version:           28.3.1
 API version:       1.41 (downgraded from 1.51)
 Go version:        go1.24.4
 Git commit:        38b7060
 Built:             Wed Jul  2 20:56:22 2025
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          20.10.24
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.19.7
  Git commit:       5d6db84
  Built:            Tue Apr  4 18:23:02 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.6.20
  GitCommit:        2806fc1057397dbaeefbea0e4e17bddfbd388f38
 runc:
  Version:          1.1.5
  GitCommit:        v1.1.5-0-gf19387a6
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
What happened?

The JVM hangs intermittently (but very often) when starting a container and TLS communication is enabled.

try (GenericContainer<?> c = new GenericContainer<>(DockerImageName.parse("alpine:latest"))) {
                    c.withExposedPorts(80);
                    c.withCommand("/bin/sh", "-c", "while true; do echo '40' | nc -l -p 80; done");
                    c.start();
}

The stacktrace from the main thread that is stuck is shown below:

"main" #1 prio=5 os_prio=0 cpu=653.31ms elapsed=19.41s tid=0x0000799864024270 nid=0x2a172 waiting on condition  [0x000079986c3fc000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@17.0.4.1/Native Method)
	- parking to wait for  <0x00000004589118a0> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
	at java.util.concurrent.locks.LockSupport.park(java.base@17.0.4.1/LockSupport.java:211)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(java.base@17.0.4.1/AbstractQueuedSynchronizer.java:715)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(java.base@17.0.4.1/AbstractQueuedSynchronizer.java:938)
	at java.util.concurrent.locks.ReentrantLock$Sync.lock(java.base@17.0.4.1/ReentrantLock.java:153)
	at java.util.concurrent.locks.ReentrantLock.lock(java.base@17.0.4.1/ReentrantLock.java:322)
	at sun.security.ssl.SSLSocketImpl$AppInputStream.read(java.base@17.0.4.1/SSLSocketImpl.java:1037)
	at org.testcontainers.shaded.okio.Okio$2.read(Okio.java:140)
	at org.testcontainers.shaded.okio.AsyncTimeout$2.read(AsyncTimeout.java:237)
	at org.testcontainers.shaded.okio.RealBufferedSource.indexOf(RealBufferedSource.java:358)
	at org.testcontainers.shaded.okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:230)
	at org.testcontainers.shaded.okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:224)
	at org.testcontainers.shaded.okhttp3.internal.http1.Http1ExchangeCodec$ChunkedSource.readChunkSize(Http1ExchangeCodec.java:489)
	at org.testcontainers.shaded.okhttp3.internal.http1.Http1ExchangeCodec$ChunkedSource.read(Http1ExchangeCodec.java:471)
	at org.testcontainers.shaded.okhttp3.internal.Util.skipAll(Util.java:204)
	at org.testcontainers.shaded.okhttp3.internal.Util.discard(Util.java:186)
	at org.testcontainers.shaded.okhttp3.internal.http1.Http1ExchangeCodec$ChunkedSource.close(Http1ExchangeCodec.java:511)
	at org.testcontainers.shaded.okio.ForwardingSource.close(ForwardingSource.java:43)
	at org.testcontainers.shaded.okhttp3.internal.connection.Exchange$ResponseBodySource.close(Exchange.java:313)
	at org.testcontainers.shaded.okio.RealBufferedSource.close(RealBufferedSource.java:476)
	at org.testcontainers.shaded.okhttp3.internal.Util.closeQuietly(Util.java:139)
	at org.testcontainers.shaded.okhttp3.ResponseBody.close(ResponseBody.java:192)
	at org.testcontainers.shaded.okhttp3.Response.close(Response.java:290)
	at org.testcontainers.shaded.com.github.dockerjava.okhttp.OkDockerHttpClient$OkResponse.close(OkDockerHttpClient.java:285)
	at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.lambda$null$0(DefaultInvocationBuilder.java:272)
	at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder$$Lambda$226/0x0000000800f25be0.close(Unknown Source)
	at com.github.dockerjava.api.async.ResultCallbackTemplate.close(ResultCallbackTemplate.java:77)
	at org.testcontainers.utility.ResourceReaper.start(ResourceReaper.java:205)
	at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:205)
	- locked <0x000000041b804b40> (a [Ljava.lang.Object;)
	at org.testcontainers.LazyDockerClient.getDockerClient(LazyDockerClient.java:14)
	at org.testcontainers.LazyDockerClient.authConfig(LazyDockerClient.java:12)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:310)
	at TestGenericContainerStart.testStartHangs(TestGenericContainerStart.java:29)

In my default docker configuration, TLS was not enabled, so in order to reproduce the problem easily, I resorted to Docker-in-Docker.

I debugged and analyzed the problem and it turns out that the hung was caused by a bug in the JDK, namely JDK-8315422.

Although, the RCA is done and no action is required from testcontainers perspective at this stage I felt it is worth logging the problem here in case other users of the testcontainers library hit this in the future.

Relevant log output

Additional Information

Full repro instructions and detailed RCA for the problem can be found under:
https://github.com/zabetak/testcontainers-tls-hang-issue-10454

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read org.testcontainers.utility.ResourceReaper.start and the call path through DockerClientFactory.client, using the supplied stack trace as the entry point. Compare the behavior with JDK-8315422 and the linked reproduction; the report states that the root cause is in the JDK and that no Testcontainers change is currently required.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, java
Domain
devtools, testing-qa
Issue type
Bug
Difficulty
1/5
Estimated time
Under an hour
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
10/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.