nextflow-io / nextflow-io/nextflow
Using "docker.enabled = true" affects availability of conda command in conda-based wave containers
Nobody has claimed this yet.
- Dominant language
- Groovy
- Stars
- 3.5k
- Forks
- 811
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 61
Description
Bug report
Expected behavior and actual behavior
We use a mix of public Docker images and YAML-defined conda environments. It appears that enabling Docker (docker.enabled = true) makes the "conda" command in yaml-defined conda environments unavailable. This is unexpected behavior to me. I was hoping to use the "conda" command to e.g. retrieve additional information about the actually installed packages in the YAML-defined environment.
Steps to reproduce the problem
With Docker disabled:
main.nf
conda.enabled = true
//docker.enabled = true
wave.enabled = true
wave.strategy = 'conda,container'
process {
withLabel: 'conda_samtools' {
conda = "$projectDir/samtools.yaml"
}
}
nextflow.config
process test_conda {
label 'conda_samtools'
debug = true
script:
"""
echo "Conda env"
samtools --version-only
echo "---"
"""
}
process test_conda2 {
label 'conda_samtools'
debug = true
script:
"""
echo "Use conda"
conda list samtools
echo "---"
"""
}
workflow {
test_conda()
test_conda2()
}
With Docker enabled:
main.nf
conda.enabled = true
docker.enabled = true
wave.enabled = true
wave.strategy = 'conda,container'
process {
withLabel: 'conda_samtools' {
conda = "$projectDir/samtools.yaml"
}
}
Program output
With Docker disabled:
N E X T F L O W ~ version 25.04.7
Launching `/main.nf` [jovial_salas] DSL2 - revision: 819790a717
executor > local (2)
[ad/a2b1cc] process > test_conda [100%] 1 of 1 ?
[d7/26b866] process > test_conda2 [100%] 1 of 1 ?
Conda env
1.22.1+htslib-1.22.1
---
Use conda
# packages in environment at /nextflow/work/conda/env-0937c70d5d6fa28d-700d5afef5901d27aac78c3b847fed0e:
#
# Name Version Build Channel
samtools 1.22.1 h96c455f_0 bioconda
With Docker enabled:
N E X T F L O W ~ version 25.04.7
Launching `/main.nf` [exotic_carson] DSL2 - revision: cec9713e8c
executor > local (2)
[75/5dffc6] process > test_conda [100%] 1 of 1 ?
[8f/480362] process > test_conda2 [ 0%] 0 of 1
Conda env
1.22.1+htslib-1.22.1
---
executor > local (2)
[75/5dffc6] process > test_conda [100%] 1 of 1 ?
[8f/480362] process > test_conda2 [ 0%] 0 of 1 ?
Conda env
1.22.1+htslib-1.22.1
---
ERROR ~ Error executing process > 'test_conda2'
Caused by:
Process `test_conda2` terminated with an error exit status (127)
Command executed:
echo "Use conda"
conda list samtools
echo "---"
Command exit status:
127
Command output:
Use conda
Command error:
Unable to find image 'wave.seqera.io/wt/c0f876d019bd/wave/build:samtools-1.22.1--0a2cf3e789c10533' locally
samtools-1.22.1--0a2cf3e789c10533: Pulling from wt/c0f876d019bd/wave/build
Digest: sha256:a552e47e0624c4967635df07ef1569d899300f77f091d5bc8dabeca6019fa6ce
Status: Downloaded newer image for wave.seqera.io/wt/c0f876d019bd/wave/build:samtools-1.22.1--0a2cf3e789c10533
Use conda
.command.sh: line 3: conda: command not found
Work dir:
/nextflow/work/8f/480362a81b65ffc1a001a92e27168f
Container:
wave.seqera.io/wt/c0f876d019bd/wave/build:samtools-1.22.1--0a2cf3e789c10533
Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line
-- Check '.nextflow.log' file for details
Environment
- Nextflow version: 25.04.7
- Java version: openjdk 17.0.10-internal 2024-01-16
- Operating system: Linux
- Bash version: GNU bash, version 5.1.8(1)-release (x86_64-redhat-linux-gnu)
Additional context
(Add any other context about the problem here)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the two configurations in main.nf and nextflow.config, comparing the commands run with Docker disabled and enabled. Inspect the reported .nextflow.log and container details to trace why conda list fails with exit status 127. Done means the YAML-defined conda environment still permits the conda command when Docker and Wave are enabled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, groovy
- Domain
- devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100