nextflow-io / nextflow-io/nextflow

envWhitelist not working with `docker.sudo = true`

Open
#5,309 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

software/docker stale
Dominant language
Groovy
Stars
3.5k
Forks
811
Avg merge
2d 11h
Merged PRs (30d)
61

Description

Bug report

Expected behavior and actual behavior

When running nextflow with docker.sudo=true and any env var in envWhitelist I expect the env vars to be available inside the process containers, however they are not, see additional context for more details.

Steps to reproduce the problem

Open your nextflow config and write

docker {
    enabled = true
    sudo = true
    envWhitelist = 'AVARTHATSHOULDTOTALLYBETHERE'
}

Now run this pipeline

#!/usr/bin/env nextflow

process testenv {
    container 'ubuntu'

    output:
    path 'out'

    """
    echo "IS IT THERE \${AVARTHATSHOULDTOTALLYBETHERE}" > out
    """
}

workflow {
    testenv | view
}
export AVARTHATSHOULDTOTALLYBETHERE="Hello, I'm here"
nextflow run ./test.nf
Program output
ERROR ~ Error executing process > 'testenv'

Caused by:
  Process `testenv` terminated with an error exit status (1)


Command executed:

  echo "IS IT THERE ${AVARTHATSHOULDTOTALLYBETHERE}" > out

Command exit status:
  1

Command output:
  (empty)

Command error:
  .command.sh: line 2: AVARTHATSHOULDTOTALLYBETHERE: unbound variable

Work dir:
  /tmp/test-nf/work/f6/98c929c696d8b31f6f12f7422a90c4

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

 -- Check '.nextflow.log' file for details
Environment
  • Nextflow version: nextflow version 24.04.4.5917
  • Java version:
openjdk 21 2023-09-19 LTS
OpenJDK Runtime Environment Temurin-21+35 (build 21+35-LTS)
OpenJDK 64-Bit Server VM Temurin-21+35 (build 21+35-LTS, mixed mode, sharing)
  • Operating system: Linux 6.8.0-44-generic #44-Ubuntu SMP PREEMPT_DYNAMIC Tue Aug 13 13:35:26 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
  • Bash version: zsh 5.9 (x86_64-ubuntu-linux-gnu)
Additional context

As a best practice, when you install the Docker daemon the installation process suggests you to not allow access to /var/run/docker.sock to your otherwise unprivileged user on the machine. This is because when a user gets added to the docker group it essentially becomes root since things like this can be done docker run -it --privileged --net=host --pid=host debian nsenter -t 1 -n -m -p.

Following this best practice, one can configure Nextflow to ask for privileges when running docker in this way:

docker {
    enabled = true
    sudo = true
    envWhitelist = 'AVARTHATSHOULDTOTALLYBETHERE'
}

This works well indeed, however when used in combination with docker.envWhitelist. Why? Because envWhitelist will share the environment variables with the shell that is starting the sudo command here but the final docker run command does not get the env vars in its /proc/self/environ because they are not automatically passed down by sudo. A possible solution to this problem is to run with sudo -E, however a safer approach would be to forward only the whitelisted env vars.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked DockerBuilder.groovy code around the sudo command and reproduce the issue using the provided Nextflow config, pipeline, and environment variable. Trace how docker.envWhitelist reaches docker run when docker.sudo is enabled; done means the whitelisted variable is available inside the container without broadly forwarding the environment.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, groovy
Domain
devops, infrastructure
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.