Docker: deprecate docker.userEmulation and use docker.runOptions; mitigates Trim Galore pipe freeze
Nobody has claimed this yet.
- Dominant language
- Nextflow
- Stars
- 229
- Forks
- 141
- Avg merge
- 13h 41m
- Merged PRs (30d)
- 1
Description
Description of the bug
Summary
When running the pipeline with -profile docker, two related issues occur:
- Deprecated config: Nextflow logs a warning that
docker.userEmulationis no longer supported and should be removed from the config. - Trim Galore can freeze: The Trim Galore step sometimes hangs in the cutadapt stage when run inside Docker. This is a known upstream issue (Trim Galore #207) caused by small pipe buffers in container environments.
Root cause (Trim Galore freeze)
- By default, Docker containers run as root (UID 0). The Linux kernel accounts pipe buffer usage per UID.
- When the total pipe usage for root exceeds
pipe-user-pages-soft, new pipes are limited to 2 pages (8192 bytes) instead of the default 16 pages (~64 KB). - With many Nextflow tasks (each creating pipes, e.g. cutadapt ↔ trim_galore), root’s quota is easily exceeded, so new pipes get the small size and Trim Galore can block waiting for the last few lines from cutadapt (see Trim Galore #207).
- Running containers as the host user (non-root) spreads pipe usage by UID and often avoids hitting that limit, which can mitigate or avoid the freeze.
Current config (problematic)
In nextflow.config, the docker profile uses the deprecated option:
docker {
docker.enabled = true
docker.userEmulation = true // deprecated, no longer supported by Nextflow
...
}
Suggested change
Remove docker.userEmulation = true from the docker profile (to clear the warning and stop relying on removed behaviour).
Add docker.runOptions = '-u $(id -u):$(id -g)' to the docker profile so containers run as the current user.
Command used and terminal output
nextflow -bg -log test.log -q run ~/pipeline/nf-core/atacseq-2.1.2 -profile docker -params-file params.json -resume
Relevant files
No response
System information
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Open nextflow.config and inspect the docker profile. Remove docker.userEmulation and add the suggested docker.runOptions value, then run the provided Nextflow command to confirm the deprecation warning is gone and the Docker tasks use the current user.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker
- Domain
- devops
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100