testcontainers / testcontainers/testcontainers-python
Bug: Reaper reports success while the session filter never reaches Ryuk (silent leak on Linux/docker-proxy)
还没有人认领这个 Issue。
- 主要语言
- Python
- 星标
- 2.3k
- 派生
- 386
- 平均合并
- 4 小时 40 分钟
- 30 天内合并 PR
- 1
描述
Describe the bug
Reaper._create_instance() can return a fully "successful" reaper whose session filter never reached Ryuk. Nothing raises, Reaper._instance is set, the ryuk container is running — and yet nothing is ever reaped. When the test process is killed (CI timeout, cancelled job, OOM), every container of that session survives.
Three things combine (testcontainers/core/container.py, 4.14.2):
- The wait strategy is registered after the container is started, so it never applies.
Reaper._container = (
DockerContainer(c.ryuk_image)
...
.start() # <- started here
)
rc = Reaper._container
rc.waiting_for(LogMessageWaitStrategy(r".* Started!").with_startup_timeout(20)) # <- too late
waiting_for is a builder setter — its own docstring says "Set a wait strategy to be used after container start" — so setting it on an already-started container has no effect. There is no wait for Started! in practice.
docker-proxyaccepts on the published port from container creation, before the ryuk process inside has bound to it. The connect loop only retries onConnectionRefusedError/OSError, so a connection that is accepted by the proxy and then reset is indistinguishable from success:
for _ in range(50):
try:
s.connect((container_host, container_port))
last_connection_exception = None
break # first attempt succeeds against the proxy
except (ConnectionRefusedError, OSError) as e:
...
- The filter is sent and the reply is never read. Ryuk answers
ACKfor every accepted filter line; nothing checks for it:
rs.send(f"label={LABEL_SESSION_ID}={SESSION_ID}\r\n".encode())
Reaper._instance = Reaper()
So the filter goes into a socket that is not (yet) the reaper, the peer resets it, and the library reports success.
Observed result
Ryuk's own log tells the story — no client ever registered:
Pinging Docker...
Docker daemon is available!
Starting on port 8080...
Started!
Timeout waiting for connection
Removed 0 container(s), 0 network(s), 0 volume(s), 0 image(s)
There is no New client connected line, and after the run is killed its containers stay up indefinitely. This is how a CI runner accumulated 40 leftover containers, including a full set of live storages still running hours after the run that created them had ended.
Why this is easy to miss
The failure is silent and platform-dependent. On macOS/Docker Desktop the published port does not accept connections before the process inside binds, so the library wins the race and everything looks correct. On a Linux daemon with docker-proxy it loses.
Measured: 5 out of 5 runs affected on a self-hosted Linux CI runner (Ubuntu in WSL2, Docker CE 29.1.3), 0 out of 5 on macOS with Docker Desktop, same library version and same code.
To Reproduce
On a Linux host with Docker CE:
from testcontainers.core.container import Reaper
Reaper.get_instance()
print("instance:", Reaper._instance) # not None — looks fine
print(Reaper._container.get_logs()[0].decode()) # no "New client connected"
# ask the socket whether anyone is on the other end
import socket
s = Reaper._socket
s.settimeout(2)
s.send(b"label=org.testcontainers.session-id=probe\r\n")
print(s.recv(64)) # ConnectionResetError instead of b"ACK"
Then start any container in a child process and SIGKILL the process: the container is still there a minute later, and ryuk exits with Removed 0 container(s).
Suggested fix
Two independent halves, either of which closes the hole:
- apply the wait strategy before connecting — e.g. move
waiting_for(...)above.start(), or explicitly wait for theStarted!log line after starting; - read Ryuk's reply after sending the filter and treat a missing
ACKas "not connected to the reaper", retrying the connection. This is the stronger of the two, because it verifies the property that actually matters — the reaper accepted the filter — rather than a proxy for it.
Runtime environment
- testcontainers-python 4.14.2
- ryuk 0.8.1 and 0.11.0 (both affected)
- Docker CE 29.1.3 on Ubuntu (WSL2) — affected; Docker Desktop on macOS — not affected
Related but different
#1093 describes the case where s.connect(...) raises and the failure is visible. This report is the opposite: the connection succeeds, no exception is raised, and the reaper silently does nothing.
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
调研方向
失败路径位于 testcontainers/core/container.py,尤其是 Reaper._create_instance() 和 socket 设置部分。首先跟踪 Linux/docker-proxy 上的 wait-strategy 注册和过滤器发送,然后使用 Reaper.get_instance() 和提供的 ACK 检查进行复现。完成的标准是 Ryuk 确认过滤器,并且 SIGKILL 不会留下任何会话容器。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- docker, python
- 领域
- devops, testing-qa
- Issue 类型
- 缺陷
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 活跃度
- 活跃
- 描述清晰度
- 描述清楚
- 新手友好度
- 68/100