Number of cores calculation for Trimgalore
Nobody has claimed this yet.
- Dominant language
- Nextflow
- Stars
- 230
- Forks
- 142
- Avg merge
- 13h 41m
- Merged PRs (30d)
- 1
Description
Description of the bug
As discussed in this issue, the calculation for actual number of cores used by Trimgalore is convoluted. The current calculation is present in this code. But, the actual core usage seems to be more than the number provided by task.cpus when the parameter --gzip is active with python3 and pigz available.
Breaking it down and drawing parallel to the comment by @FelixKrueger in the same thread, in the trimming section,
- cutadapt uses cores provided by the
-jargument - trim_galore: 1 (high)
- write to pigz stream: 1 (low)
- however, pigz would also use
-jcores
This would mean, at least 2 * j + 1 cores for the trimming process.
The same would extend to the validation stage where due to pigz, the calculation would include additional j cores than the prior calculation.
Command used and terminal output
Relevant files
No response
System information
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked nf-core/trimgalore main.nf calculation around line 30 and review how the trimming and validation stages account for cutadapt, trim_galore, and pigz. Done means the resource calculation reflects the additional cores used when gzip, Python 3, and pigz are active.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- bioinformatics
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100