testcontainers / testcontainers/testcontainers-java

[Enhancement]: Being able to use Ryuk in (CI) environment where Docker "userns-remap" mode is active

Open
#7,964 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type/enhancement
Dominant language
Java
Stars
8.7k
Forks
1.9k
Avg merge
2d 17h
Merged PRs (30d)
9

Description

Module

Core

Proposal

Hello,

I'm using Testcontainers in an on-prem GitLab CI environment, where the GitLab runner hosts have their Docker server with userns-remap turned on.
I ran into a couple of issues related to Ryuk in this case.
First, from what I read on https://java.testcontainers.org/features/configuration/#customizing-ryuk-resource-reaper, the Ryuk container must be started as a privileged container.
That's the reason why I get the following error when Testcontainers tried launching the Ryuk container:

...
com.github.dockerjava.api.exception.BadRequestException: Status 400: {"message":"privileged mode is incompatible with user namespaces.  You must run the container in the host namespace when running privileged mode"}
...

As a quick hack, I worked around this issue by "monkeypatching" org.testcontainers.utility.RyukContainer and adding:

    cmd
        .getHostConfig()
        ...
        .withUsernsMode("host")  /// <= added

The Ruyk container could then start fine.

But due to the fact the GitLab runner container conforms to this userns-remap mode (and thus not Ruyk as per the above), Testcontainers was not able to connect to Ryuk:

...
WARN --- [containers-ryuk] o.t.utility.RyukResourceReaper : Can not connect to Ryuk at 172.17.0.1:35592

Again, I had to resort to monkeypatching org.testcontainers.utility.RuykResourceReaper to have:

...
// String host = ryukContainer.getHost();
String host = ryukContainer.getCurrentContainerInfo().getNetworkSettings().getIpAddress();
// Integer ryukPort = ryukContainer.getFirstMappedPort();
Integer ryukPort = ryukContainer.getExposedPorts().get(0);
...

My understanding is that this fix is obviously not portable in environments where the Docker host is not local to where Testcontainers is running.
But bar changing my CI environment setup to drop the userns-remap mode, I don't see any other solution to my issue.

What do you think of having "official support" for such a workaround in Testcontainers, so I don't have to monkeypatch Java classes... 🤓
If not via a setting (e.g. ryuk.container.userns_mode=host or something related), maybe via a way to customize the behavior of Testcontainers dynamically?

Thanks!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review org.testcontainers.utility.RyukContainer and RyukResourceReaper, along with the custom Ryuk configuration documentation. Trace how the container host namespace and connection address are selected, then determine how an official option could support Docker userns-remap without breaking remote Docker environments. Done should include tests covering the supported configuration and connection behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, java
Domain
devops, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.