QUANT: silent per-sample drop from order-dependent combine(by:) on groupKey
Nobody has claimed this yet.
- Dominant language
- Nextflow
- Stars
- 51
- Forks
- 35
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 7
Description
Description of the bug
In QUANT (subworkflows/local/quant/main.nf), runs are matched to their group's FDR-filter file with:
ch_runs_score_switched
.map { meta, idxml -> [groupKey([id: "${meta.sample}_${meta.condition}"], meta.group_count), meta, idxml] }
.combine(filter_q_value, by:0)
The left key is a groupKey(...) object; filter_q_value's key is a plain [id: ...] map. groupKey's equals() is asymmetric (groupKey(v,n) == v is true, but v == groupKey(v,n) is false, see nextflow-io/nextflow#4104). So whether a pair matches depends on which operand is compared first, which depends on the arrival order of OPENMS_IDRIPPER vs OPENMS_IDFILTER_Q_VALUE.
When IDRipper arrives first, the pair is silently dropped — no error, pipeline exits 0. Under executor dispatch jitter (e.g. AWS Batch) this drops a large fraction of samples at QUANT:OPENMS_IDFILTER_QUANT.
Reproduced on Nextflow 25.10.4: forcing IDRipper-before-IDFILTER_Q_VALUE drops the sample 5/5 runs; reversing the order keeps it. Matches the field report in #458 (Failure A).
CI misses it: test is one group, test_single_quant is small, and test_full asserts only that the run finishes.
Fix: drop the groupKey from the match key and combine on a plain [id: ...] map (symmetric equality). The groupKey size isn't needed here — the key is discarded right after the combine.
Command used and terminal output
nextflow run nf-core/mhcquant -r 3.2.0 -profile docker --quantify true ...
# multi-sample, group_count=1 samplesheet on awsbatch
# QUANT:OPENMS_IDFILTER_QUANT receives far fewer tasks than QUANT:OPENMS_IDRIPPER, no error
Relevant files
subworkflows/local/quant/main.nf
System information
- Nextflow: 25.10.4
- Executor: awsbatch (also reproducible locally by forcing task arrival order)
- Container: Docker
- nf-core/mhcquant: 3.2.0
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in subworkflows/local/quant/main.nf by reading the ch_runs_score_switched mapping and its combine with filter_q_value. Reproduce the issue by forcing IDRipper and IDFILTER_Q_VALUE arrival orders, then verify that all expected samples reach QUANT:OPENMS_IDFILTER_QUANT regardless of order and that the workflow completes successfully.
Written by the indexing model from the issue text.
Assessment
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100