Process writes to host home directory which is mounted in the container
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 322
- Forks
- 255
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 5
Description
Hi, within the epitopeprediction pipeline, the tool mhcflurry tries to create a folder within the HOME directory to store some downloaded data:
-
With the
Dockerprofile on Mac this causes apermission deniederror, becausemhcflurrytries to create the folder/.local(I am not sure, why in this case it does not use the home directory, @apeltzer mentioned it could be a Mac specific problem). In the past, for some pipelines the same problem was addressed by uncommenting thedocker.runOptions = '-u \$(id -u):\$(id -g)'line in thenextflow.config(https://github.com/nf-core/tools/pull/351, https://github.com/nf-core/mhcquant/pull/104). By this, the commands will be executed as root again, preventing this particularpermission deniederror, but maybe also giving raise to other errors again (https://github.com/nf-core/tools/pull/351). (Another solution for this particular problem is https://github.com/nf-core/epitopeprediction/pull/52) -
With Singularity the folder
.localis created in the actual hostHOMEdirectory, and everything seems smooth on the first view.
However, the fact that the host HOME directory is included into the container, allowing tools to actually write to and read from it, potentially unnoticed, is from a reproducibility perspective not ideal and should probably be avoided.
See also a longer discussion by @lkuchenb, @drpatelh, @pontus and @pditommaso about this topic on the slack help channel.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Docker profile and the docker.runOptions setting in nextflow.config, then compare its behavior with Singularity and the linked epitopeprediction solution. Done means containerized tools cannot unexpectedly read from or write to the host HOME directory while the affected pipeline still runs reproducibly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker
- Domain
- devops, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100