nf-core / nf-core/modules

[FEATURE] Update modules to remove undocumented task.ext fields

Open
#8,716 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

good first issue
Dominant language
Nextflow
Stars
429
Forks
1.1k
Avg merge
1d 6h
Merged PRs (30d)
153

Description

Is your feature request related to a problem? Please describe

task.ext fields were introduced allow users to override developer settings in pipelines. However some modules are simply using them as an alternative input: method for parameters that don't always need to be supplied.

% find modules -name "*.nf" -exec grep -H "task.ext." {} \; | grep -vE "task\.ext\.(singularity|args|prefix|when|suffix)"
modules/nf-core/picard/fixmateinformation/main.nf:    def STRINGENCY = task.ext.stringency ?: "STRICT"
modules/nf-core/artic/minion/main.nf:    def hd5_plugin_path = task.ext.hd5_plugin_path ? "export HDF5_PLUGIN_PATH=${task.ext.hd5_plugin_path}" : "export HDF5_PLUGIN_PATH=/usr/local/lib/python3.6/site-packages/ont_fast5_api/vbz_plugin"
modules/nf-core/fgbio/zipperbams/main.nf:    def compression = task.ext.compression ?: '0'
modules/nf-core/jupyternotebook/main.nf:    def parametrize = (task.ext.parametrize == null) ?  true : task.ext.parametrize
modules/nf-core/jupyternotebook/main.nf:    def implicit_params = (task.ext.implicit_params == null) ? true : task.ext.implicit_params
modules/nf-core/jupyternotebook/main.nf:    def meta_params = (task.ext.meta_params == null) ? true : task.ext.meta_params
modules/nf-core/jupyternotebook/main.nf:    def kernel   = task.ext.kernel ?: '-'
modules/nf-core/rmarkdownnotebook/main.nf:    def parametrize = (task.ext.parametrize == null) ?  true : task.ext.parametrize
modules/nf-core/rmarkdownnotebook/main.nf:    def implicit_params = (task.ext.implicit_params == null) ? true : task.ext.implicit_params
modules/nf-core/rmarkdownnotebook/main.nf:    def meta_params = (task.ext.meta_params == null) ? true : task.ext.meta_params
modules/nf-core/plink2/vcf2bgen/main.nf:    def reffirst = bgen_reffirst || task.ext.reffirst ? "ref-first" : ""
modules/nf-core/plink2/vcf2bgen/main.nf:    def dosage = task.ext.dosage_field ? task.ext.dosage_field : dosage_field
modules/nf-core/plink2/vcf2bgen/main.nf:    def sample_name_opt = task.ext.sample_name_mode ? task.ext.sample_name_mode : sample_name_mode
modules/nf-core/hmmer/eslreformat/main.nf:    def postproc = task.ext.postprocessing ?: ""
modules/nf-core/cellbender/merge/main.nf:    output_layer = task.ext.output_layer ?: "cellbender"
modules/nf-core/cellbender/removebackground/main.nf:    container "${ task.ext.use_gpu ? 'us.gcr.io/broad-dsde-methods/cellbender:0.3.2' :
modules/nf-core/cellbender/removebackground/main.nf:    use_gpu = task.ext.use_gpu ? "--cuda" : ""
modules/nf-core/bedtools/groupby/main.nf:    def summary_col = task.ext.summary_col ? "-c ${task.ext.summary_col}" : "-c 5"
modules/nf-core/scvitools/scar/main.nf:    input_layer = task.ext.input_layer ?: "X"
modules/nf-core/scvitools/scar/main.nf:    output_layer = task.ext.output_layer ?: "scar"
modules/nf-core/scvitools/scar/main.nf:    max_epochs = task.ext.max_epochs ?: ""
modules/nf-core/scvitools/solo/main.nf:    container "${ task.ext.use_gpu ? 'docker.io/nicotru/scvitools-gpu:cuda-12' :
modules/nf-core/scvitools/solo/main.nf:    batch_key = task.ext.batch_key ?: ""
modules/nf-core/scvitools/solo/main.nf:    max_epochs = task.ext.max_epochs ?: ""
modules/nf-core/metaspace/converter/main.nf:    database_name    = task.ext.database_name ?: "HMDB"
modules/nf-core/metaspace/converter/main.nf:    database_version = task.ext.database_version ?: "v4"
modules/nf-core/metaspace/converter/main.nf:    fdr              = task.ext.fdr ?: "0.1"
modules/nf-core/metaspace/converter/main.nf:    use_tic          = task.ext.use_tic ?: "true"
modules/nf-core/metaspace/converter/main.nf:    metadata_as_obs  = task.ext.metadata_as_obs ?: "false"
modules/nf-core/sentieon/dedup/main.nf:    def metrics = task.ext.metrics ?: "${prefix}.metrics"
modules/nf-core/sentieon/dedup/main.nf:    def metrics = task.ext.metrics ?: "${prefix}.metrics"
modules/nf-core/icountmini/segment/main.nf:    def regions_prefix = task.ext.regions_prefix ?: "${gtf.simpleName}"
modules/nf-core/icountmini/segment/main.nf:    def regions_prefix = task.ext.regions_prefix ?: "${gtf.simpleName}"
Describe the solution you'd like

The modules should be refactored so the "non-standard" task.ext fields are moved to input: and documented in the meta.yml so pipeline developers know what to set.

Describe alternatives you've considered

No response

Additional context

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the listed modules/*/main.nf files and their corresponding meta.yml files, using the grep command in the issue to inventory non-standard task.ext fields. Check how each field is currently consumed and how module inputs are documented. Done means the listed fields are exposed through input: and documented in meta.yml, with no undocumented non-standard task.ext usage remaining.

Written by the indexing model from the issue text.

Assessment

Domain
tooling
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.