nf-core / nf-core/mhcquant

QUANT: silent per-sample drop from order-dependent combine(by:) on groupKey

Open Beginner friendly
#459 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Nextflow
Stars
51
Forks
35
Avg merge
1d 10h
Merged PRs (30d)
7

Description

Description of the bug

In QUANT (subworkflows/local/quant/main.nf), runs are matched to their group's FDR-filter file with:

ch_runs_score_switched
    .map { meta, idxml -> [groupKey([id: "${meta.sample}_${meta.condition}"], meta.group_count), meta, idxml] }
    .combine(filter_q_value, by:0)

The left key is a groupKey(...) object; filter_q_value's key is a plain [id: ...] map. groupKey's equals() is asymmetric (groupKey(v,n) == v is true, but v == groupKey(v,n) is false, see nextflow-io/nextflow#4104). So whether a pair matches depends on which operand is compared first, which depends on the arrival order of OPENMS_IDRIPPER vs OPENMS_IDFILTER_Q_VALUE.

When IDRipper arrives first, the pair is silently dropped — no error, pipeline exits 0. Under executor dispatch jitter (e.g. AWS Batch) this drops a large fraction of samples at QUANT:OPENMS_IDFILTER_QUANT.

Reproduced on Nextflow 25.10.4: forcing IDRipper-before-IDFILTER_Q_VALUE drops the sample 5/5 runs; reversing the order keeps it. Matches the field report in #458 (Failure A).

CI misses it: test is one group, test_single_quant is small, and test_full asserts only that the run finishes.

Fix: drop the groupKey from the match key and combine on a plain [id: ...] map (symmetric equality). The groupKey size isn't needed here — the key is discarded right after the combine.

Command used and terminal output
nextflow run nf-core/mhcquant -r 3.2.0 -profile docker --quantify true ...
# multi-sample, group_count=1 samplesheet on awsbatch
# QUANT:OPENMS_IDFILTER_QUANT receives far fewer tasks than QUANT:OPENMS_IDRIPPER, no error
Relevant files

subworkflows/local/quant/main.nf

System information
  • Nextflow: 25.10.4
  • Executor: awsbatch (also reproducible locally by forcing task arrival order)
  • Container: Docker
  • nf-core/mhcquant: 3.2.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in subworkflows/local/quant/main.nf by reading the ch_runs_score_switched mapping and its combine with filter_q_value. Reproduce the issue by forcing IDRipper and IDFILTER_Q_VALUE arrival orders, then verify that all expected samples reach QUANT:OPENMS_IDFILTER_QUANT regardless of order and that the workflow completes successfully.

Written by the indexing model from the issue text.

Assessment

Domain
data-engineering
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.