nf-core / nf-core/eager

DSL2: Only load modules if being used

Open
#830 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

DSL2 enhancement low_priority
Dominant language
Nextflow
Stars
215
Forks
91
Avg merge
5m
Merged PRs (30d)
1

Description

I realised that in DSL2 there is possibly one way to reduce the size of the console logging of all the processes (which is currently very large for eager, and ends up a user losing any warnings at the top of the very long list of processes).

We could maybe not have these displayed if we only include those modules when they are requested.

e.g.

if ( params.run_deeparg ) {
    include { DEEPARG_DOWNLOADDATA } from '../modules/nf-core/modules/deeparg/downloaddata/main'
    include { DEEPARG_PREDICT      } from '../modules/nf-core/modules/deeparg/predict/main'
}

Note: each invocation of those modules will also need to be wrapped in params.run_<tool>

if ( params.run_deeparg ) {
    DEEPARG_PREDICT ( ch_input_for_deeparg, ch_deeparg_db )
    ch_versions = ch_versions.mix(DEEPARG_PREDICT.out.versions)
}

Current issue: get a

WARN: There's no process matching config selector: DEEPARG_PREDICT

But maybe this can also be invoked conditionally within modules.config?

And otherwise, DEEPARG is thus not displayed unless --run_deeparg is supplied

[a6/ea47cb] process > NFCORE_FUNCSCAN:FUNCSCAN:INPUT_CHECK:SAMPLESHEET_CHECK (input.csv) [100%] 1 of 1 ✔
[-        ] process > NFCORE_FUNCSCAN:FUNCSCAN:GUNZIP                                    -
[f5/f68fbf] process > NFCORE_FUNCSCAN:FUNCSCAN:PROKKA (deeparg1)                         [100%] 1 of 1, failed: 1 ✘
[-        ] process > NFCORE_FUNCSCAN:FUNCSCAN:FARGENE                                   -
[-        ] process > NFCORE_FUNCSCAN:FUNCSCAN:CUSTOM_DUMPSOFTWAREVERSIONS               -
[-        ] process > NFCORE_FUNCSCAN:FUNCSCAN:MULTIQC                                   [  0%] 0 of 1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the DSL2 workflow's DEEPARG module includes and invocations, along with modules.config, and reproduce the current WARN: There's no process matching config selector: DEEPARG_PREDICT behavior. Done means DEEPARG processes are included and invoked only when --run_deeparg is supplied, without warnings or unused-process entries in the console output.

Written by the indexing model from the issue text.

Assessment

Domain
bioinformatics
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.