nf-core / nf-core/atacseq

Improve TRIMGALORE by adding pigz to Singularity container

Open
#394 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Nextflow
Stars
229
Forks
141
Avg merge
13h 41m
Merged PRs (30d)
1

Description

Description of feature

Dear NFcore-ATACseq team,
when running nf-core/atacseq -r 2.1.2 I noticed in the .command.log of the tasks NFCORE_ATACSEQ:ATACSEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE that this rule seems to make inefficient use of gzip trying to parallelize. A solution could be to add pigz to the Singularity container. I did not specify extra number of cores, etc.. See below the relevant part of .command.log:

====.coomand.log:
Using an excessive number of cores has a diminishing return! It is recommended not to exceed 8 cores per trimming process (you asked for 8 cores). Please consider re-specifying
Path to Cutadapt set as: 'cutadapt' (default)
Cutadapt seems to be working fine (tested command 'cutadapt --version')
Cutadapt version: 3.4
Could not detect version of Python used by Cutadapt from the first line of Cutadapt (but found this: >>>#!/bin/sh<<<)
Letting the (modified) Cutadapt deal with the Python version instead
Proceeding with 'gzip' for compression. PLEASE NOTE: Using multi-cores for trimming with 'gzip' only has only very limited effect! (see here: https://github.com/FelixKrueger/TrimGalore/issues/16#issuecomment-458557103)
To increase performance, please install 'pigz' and run again

==== call of workflow:
nextflow run nf-core/atacseq -r 2.1.2 -profile singularity -c hpc_slurm_cbg_execution.config
--input samplesheet.csv
--outdir .
--read_length 100
--fasta genome/mm39_dm6.fa
--gtf genome/mm39_dm6.gtf
--blacklist genome/mm39_dm6_bl_v2.bed
--mito_name MT
--ataqv_mito_reference MT
--trim_nextseq 20
--aligner bowtie2
--keep_dups false
--keep_multi_map false
--narrow_peak false
--min_reps_consensus 1
--save_macs_pileup true
--deseq2_vst true
--skip_deseq2_qc false

==== hpc_slurm_cbg_execution.config:
process{
executor='slurm'
queue='batch'
clusterOptions = '--ntasks=1 --time=24:00:00'
}

==== nextflow version:
nextflow version 24.04.4.5917

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Singularity container used by NFCORE_ATACSEQ:ATACSEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE and inspect its dependency definition and task log. The change is done when pigz is available in that container and the Trim Galore run selects it instead of gzip; verify this through the reported workflow task and its .command.log.

Written by the indexing model from the issue text.

Assessment

Domain
devops
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.