nextflow-io / nextflow-io/nextflow
Error handling for processes
Nobody has claimed this yet.
- Dominant language
- Groovy
- Stars
- 3.5k
- Forks
- 811
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 61
Description
We would like NF to support metadata tracking and routing; this would for example enable a process to inform another process that an action was not taken or a stage not passed, this could trigger other tasks. I include two examples from our current pipeline below; perhaps not a perfect match, but they informed my initial thinking about this.
process irods {
tag "${samplename}"
input:
val samplename from sample_list.flatMap{ it.readLines() }
output:
set val(samplename), file('*.cram') optional true into ch_cram_files
file('*.lostcause.txt') optional true into ch_lostcause_irods
script:
"""
if bash -euo pipefail irods.sh -t ${params.studyid} -s ${samplename}; then
true
else
stat=\$?
tag='UNKNOWN'
if [[ \$stat == 64 ]]; then tag='nofiles'; fi
echo -e "${samplename}\\tirods\\t\$tag" > ${samplename}.lostcause.txt
fi
"""
}
# later ...
ch_star_accept = Channel.create()
ch_star_reject = Channel.create()
star_aligned
.choice(ch_star_accept, ch_star_reject)
{ namelogsbams -> check_log(namelogsbams[1]) ? 0 : 1 }
ch_star_accept
.map { name, logs, bams -> [name, bams] }
.into { ch_featurecounts; ch_indexbam }
ch_star_reject
.map { it -> "${it[0]}\tSTAR\tlowmapping\n" }
.mix(ch_lostcause_irods)
.set { ch_lostcause }
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue provides Groovy process and channel examples but names no files or tests. Start by tracing the existing process-failure and channel-routing behavior, then clarify the metadata model and acceptance cases with maintainers. Done would require an agreed design and tests covering process status propagation and routing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- bash, groovy
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100