nextflow-io / nextflow-io/nextflow

`nextflow inspect` no longer respects container overrides from config for `withName`

Open
#6,164 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Groovy
Stars
3.5k
Forks
811
Avg merge
2d 11h
Merged PRs (30d)
61

Description

Bug report

In previous versions of nextflow inspect container overrides set using any withName selectors were respected and reported correctly. Since (at least) 25.04 this is broken and overrides only work from withName that uses only the process name (e.g. not alias, fully qualified name, partial qualified process name etc).

Expected behavior and actual behavior

For the following workflow main.nf please see the working/non-working examples.

process A_A { 

    script:
    """
    echo test
    """
}

process A_B { 

    script:
    """
    echo test
    """
}

workflow PARENT {
    A_A()
    A_B()
}

workflow {
    PARENT()
}

Working in both

Process name only

nextflow.config

process {

    withName: A_A {
        container = "docker.io/google/deepvariant:1.8.0"
    }

}

24.10.5

{
    "processes": [
        {
            "name": "PARENT:A_A",
            "container": "docker.io/google/deepvariant:1.8.0"
        }
    ]
}

25.04.3

{
    "processes": [
        {
            "name": "A_A",
            "container": "docker.io/google/deepvariant:1.8.0"
        }
    ]
}
Quoted process name only

nextflow.config

process {

    withName: "A_A" {
        container = "docker.io/google/deepvariant:1.8.0"
    }

}

24.10.5

{
    "processes": [
        {
            "name": "PARENT:A_A",
            "container": "docker.io/google/deepvariant:1.8.0"
        }
    ]
}

25.04.3

{
    "processes": [
        {
            "name": "A_A",
            "container": "docker.io/google/deepvariant:1.8.0"
        }
    ]
}
Partial process name

nextflow.config

process {

    withName: "A_.*" {
        container = "docker.io/google/deepvariant:1.8.0"
    }

}

24.10.5

nextflow inspect main.nf
{
    "processes": [
        {
            "name": "PARENT:A_B",
            "container": "docker.io/google/deepvariant:1.8.0"
        },
        {
            "name": "PARENT:A_A",
            "container": "docker.io/google/deepvariant:1.8.0"
        }
    ]
}

25.04.3

{
    "processes": [
        {
            "name": "A_B",
            "container": "docker.io/google/deepvariant:1.8.0"
        },
        {
            "name": "A_A",
            "container": "docker.io/google/deepvariant:1.8.0"
        }
    ]
}

Working only in 24.10.5

Fully qualified process name

nextflow.config

process {

    withName: "PARENT:A_A" {
        container = "docker.io/google/deepvariant:1.8.0"
    }

}

24.10.5

{
    "processes": [
        {
            "name": "PARENT:A_A",
            "container": "docker.io/google/deepvariant:1.8.0"
        }
    ]
}

25.04.3

{
    "processes": [
    ]
}
Partial qualified process name

nextflow.config

process {

    withName: ".*RENT:A_A" {
        container = "docker.io/google/deepvariant:1.8.0"
    }

}

24.10.5

{
    "processes": [
        {
            "name": "PARENT:A_A",
            "container": "docker.io/google/deepvariant:1.8.0"
        }
    ]
}

25.04.3

{
    "processes": [
    ]
}

ALIASING

For the following tests module definitions were moved to module.nf and top-level main.nf is now

include { A_A; A_A as A_B } from './module.nf'

workflow PARENT {
    A_A()
    A_B()
}

workflow {
    PARENT()
}
Alias by base name

nextflow.config

process {

    withName: A_A {
        container = "docker.io/google/deepvariant:1.8.0"
    }

}

24.10.5

{
    "processes": [
        {
            "name": "PARENT:A_B",
            "container": "docker.io/google/deepvariant:1.8.0"
        },
        {
            "name": "PARENT:A_A",
            "container": "docker.io/google/deepvariant:1.8.0"
        }
    ]
}

25.04.3

{
    "processes": [
        {
            "name": "A_A",
            "container": "docker.io/google/deepvariant:1.8.0"
        }
    ]
}
Alias by alias

nextflow.config

process {

    withName: "A_B" {
        container = "docker.io/google/deepvariant:1.8.0"
    }

}

24.10.5

{
    "processes": [
        {
            "name": "PARENT:A_B",
            "container": "docker.io/google/deepvariant:1.8.0"
        }
    ]
}

25.04.3

{
    "processes": [
        
    ]
}
Alias by alias and base name

nextflow.config

process {

    withName: "A_B" {
        container = "docker.io/google/deepvariant:1.9.0"
    }


    withName: "A_A" {
        container = "docker.io/google/deepvariant:1.8.0"
    }

}

24.10.5

{
    "processes": [
        {
            "name": "PARENT:A_B",
            "container": "docker.io/google/deepvariant:1.9.0"
        },
        {
            "name": "PARENT:A_A",
            "container": "docker.io/google/deepvariant:1.8.0"
        }
    ]
}

25.04.3

{
    "processes": [
        {
            "name": "A_A",
            "container": "docker.io/google/deepvariant:1.8.0"
        }
    ]
}

I haven't tested overrides by e.g. withLabel but as you can see it appears that since at least 25.04.3 the nextflow inspect command reports only "base modules" and not the fully qualified names/paths as in 24.10.5 and this appears to affect the correctness of the reporting.

Environment
  • Nextflow version: 25.04.3
  • Java version: openjdk version "21.0.7" 2025-04-15 LTS
  • Operating system: macOS

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the regression with nextflow inspect using the main.nf, nextflow.config, and module.nf examples in the report, comparing qualified names, partial selectors, and aliases. Trace how inspect resolves process configuration and verify that its output reports the configured container for the fully qualified and aliased processes as in 24.10.5.

Written by the indexing model from the issue text.

Assessment

Tech stack
groovy
Domain
cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.