nextflow-io / nextflow-io/nextflow
Recursion with multiple inputs hangs with `until()`
@bentsherman is already working on this.
Since Jul 11, 2022.
- Dominant language
- Groovy
- Stars
- 3.5k
- Forks
- 811
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 61
Description
Bug report
Recursion with until() works when the process/workflow has one input/output but hangs at the end when there are multiple inputs/outputs.
Steps to reproduce the problem
I have provided a few recursion examples that work, along with modified examples that have multiple inputs and don't work with until() (but do work with times()).
until.nf
nextflow.preview.recursion=true
params.data = "$baseDir/hello.txt"
process foo {
input:
path 'input.txt'
output:
path 'result.txt'
script:
"""
cat input.txt > result.txt
echo "Task $task.index added this" >> result.txt
"""
}
workflow {
foo
.recurse(file(params.data))
.until{ it-> it.size() > 100 }
foo
.out
.view(it->it.text)
}
until2.nf
nextflow.preview.recursion=true
params.data = "$baseDir/hello.txt"
process foo {
echo true
input:
val i
path 'input.txt'
output:
val i
path 'result.txt'
script:
"""
cat input.txt > result.txt
echo "Task ${i} added this" >> result.txt
cat result.txt
stat -c '%s' result.txt
# ${i = i + 1}
"""
}
workflow {
foo
.recurse(1, file(params.data))
.until{ i, f -> f.size() > 100 }
}
counter.nf
nextflow.preview.recursion=true
process count_down {
input:
val v
output:
val v
exec:
v = v - 1
}
workflow {
n = 10
count_down.recurse(n).until({ v -> v == 0 })
count_down.out[0].view()
}
counter2.nf
nextflow.preview.recursion=true
process count_down {
input:
val u
val v
output:
val u
val v
exec:
u = u + 1
v = v - 1
}
workflow {
n = 10
// count_down.recurse(0, n).times(n)
count_down.recurse(0, n).until({ u, v -> v == 0 })
count_down.out[1].view()
}
Program output
The examples until.nf and counter.nf complete successfully. The alternate examples until2.nf and counter2.nf hang at the very end. If you replace the until() line with the times() line, they complete successfully.
Environment
- Nextflow version: 21.12.1-edge
- Java version: 1.8.0_292
- Operating system: Linux
- Bash version: 4.4.19(1)-release (x86_64-pc-linux-gnu)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.