Review subworkflows: remove thin wrappers, migrate utility functions to plugins
Nobody has claimed this yet.
- Dominant language
- Nextflow
- Stars
- 429
- Forks
- 1.1k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 153
Description
Summary
We've audited all 113 subworkflows and categorized them by complexity. A number of subworkflows are thin wrappers around 1-2 tools and don't meet the bar of what a subworkflow should provide (3+ tools or reasonably complex channel operations/functions). Additionally, several utility subworkflows contain pure functions that should migrate to plugins.
This issue is for community discussion on which to remove, consolidate, or migrate.
Complexity breakdown
| Category | Count |
|---|---|
| TRIVIAL (1 tool, no logic) | 2 |
| SIMPLE (1-2 tools, minimal logic) | 27 |
| MODERATE (3+ tools or complex ops) | 52 |
| COMPLEX (3+ tools + complex logic) | 30 |
| Empty/placeholder | 2 |
MODERATE and COMPLEX subworkflows are considered justified and are not listed here.
TRIVIAL — Strong removal candidates
These are single-tool wrappers with no meaningful added logic:
-
mafft_align— wrapper around MAFFT_ALIGN with empty optional inputs -
vcf_annotate_ensemblvep— wrapper around ENSEMBLVEP_VEP with a join for vcf+tbi output
Empty/placeholder directories — Remove
-
multiple_impute_glimpse2— README only, no main.nf -
preprocess_rnaseq— README only, no main.nf
SIMPLE — Review for removal or consolidation
Single tool + sort/stats subworkflow (repeated pattern)
These all follow the same pattern: one aligner call + BAM_SORT_STATS_SAMTOOLS. Could potentially be consolidated into a single parameterized subworkflow, or users can call the aligner module + sort/stats subworkflow directly.
-
fastq_align_bowtie2— Bowtie2 + BAM_SORT_STATS_SAMTOOLS -
fastq_align_bwa— BWA MEM + BAM_SORT_STATS_SAMTOOLS -
fastq_align_chromap— Chromap + BAM_SORT_STATS_SAMTOOLS -
fastq_align_hisat2— HISAT2 + BAM_SORT_STATS_SAMTOOLS -
fastq_align_star— STAR + 2x BAM_SORT_STATS_SAMTOOLS (genome + transcriptome)
2 tools, linear pipeline (A → B)
These are simple sequential tool calls with basic channel wiring. Users could call the two modules directly.
-
bam_cnv_wisecondorx— WISECONDORX_CONVERT + PREDICT -
bam_docounts_contamination_angsd— ANGSD doCounts + contamination -
bam_methyldackel— MethylDackel mbias + extract -
bam_stringtie_merge— StringTie assembly + merge -
bedgraph_bedclip_bedgraphtobigwig— bedClip + bedGraphToBigWig -
cache_download_ensemblvep_snpeff— download VEP + SnpEff caches -
vcf_annotate_snpeff— SnpEff annotate + tabix -
fastq_fetch_clean_hostile— fetch reference + clean with hostile -
fastq_extract_kraken_krakentools— Kraken extract reads -
fastq_download_prefetch_fasterqdump_sratools— prefetch + fasterq-dump -
h5ad_removebackground_barcodes_cellbender_anndata— CellBender + AnnData -
fastq_ngscheckmate— patterngenerator + vafncm -
fastq_index_filter_deacon— Deacon index + filter -
fasta_clean_fcs— FCSGX_RUNGX + FCSADAPTOR_CLEAN -
fasta_build_add_kraken2— Kraken2 add + build -
bam_ngscheckmate— BCFtools mpileup + vafncm -
fastq_contam_seqtk_kraken— SeqTK subsample + Kraken2
Minimal logic wrappers
-
bed_scatter_bedtools— 1 tool (BEDTOOLS_SPLIT) with map + transpose -
fastq_decontaminate_deacon_hostile— pure routing to 2 other subworkflows, no tools
Sort/dedup + stats combos
-
bam_sort_stats_samtools— sort + index + BAM_STATS_SAMTOOLS -
bam_markduplicates_picard— MarkDuplicates + index + stats subworkflow -
bam_dedup_stats_samtools_umicollapse— UMI-collapse dedup + index + stats subworkflow -
quantify_pseudo_alignment— Salmon/Kallisto quant + tximport subworkflow
Utility subworkflows — Plugin migration candidates
These contain pure functions/logic with no tool invocations. Their functions should move to Nextflow plugins:
-
utils_nextflow_pipeline— version check, params dump -
utils_nfcore_pipeline— 45+ custom utility functions -
utils_annotation_cache— cache path resolution logic -
utils_references— reference file resolution with custom functions
(utils_nfschema_plugin and utils_nfvalidation_plugin are already plugin-based.)
Guiding principles
- Subworkflows should have 3+ tools or reasonably complex channel operations/functions
- Pure functions should live in plugins, not subworkflows
- Simple linear A→B pipelines can be called as two modules directly in a workflow
- Repeated patterns (align + sort/stats) could be consolidated
Next steps
Community feedback on each item above. Please comment with 👍/👎 or discussion on specific subworkflows you'd like to keep/remove/consolidate.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the listed subworkflow directories under subworkflows/nf-core, including each README.md and whether main.nf exists. Compare the wrappers, linear pipelines, and utility functions against the stated principles, then comment with a specific keep, remove, consolidate, or plugin-migration recommendation for each item; done means every candidate has a community-backed disposition.
Written by the indexing model from the issue text.
Assessment
- Domain
- tooling
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100