Bowtie2 + Samtools pipeline in nf-core module may use double the intended threads
Nobody has claimed this yet.
- Dominant language
- Nextflow
- Stars
- 429
- Forks
- 1.1k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 153
Description
Have you checked the docs?
Description of the bug
In the nf-core Bowtie2 alignment module, the pipeline runs something like:
bowtie2 ... --threads $task.cpus ... | samtools ... --threads $task.cpus ...
Because Bowtie2 and Samtools are piped, both programs seem to independently use $task.cpus threads.
I observed this when I was running nf-core/atacseq, and the bowtie_align step was using more cores than it is supposed to use. When inspecting processes with htop, I saw a bowtie2 process and a samtools process, both using 12 threads, and I had only given 12 threads to this process. I scheduled the process with slurm on our cluster.
Should this be solved by making sure that bowtie2 and samtools both ask for less threads than $task.cpus?
Command used and terminal output
Relevant files
No response
System information
nextflow 25.10.0
run on our HPC with slurm executor
container engine: singularity
CentOS Linux release 7.4
running the dev branch of nf-core/atacseq, which uses the bowtie2_align module
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read modules/nf-core/bowtie2/align/main.nf around lines 57-64, then inspect how the module assigns task.cpus to Bowtie2 and Samtools. Reproduce or verify the behavior through the nf-core/atacseq bowtie_align step under Slurm, and confirm that the module's total thread usage matches its requested allocation.
Written by the indexing model from the issue text.
Assessment
- Domain
- bioinformatics, tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100