TimelyDataflow / TimelyDataflow/timely-dataflow
Looping In Timely Dataflow
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 293
- Avg merge
- 14h 46m
- Merged PRs (30d)
- 4
Description
let probe = worker.dataflow(|scope|{
let (handle, cycle) = scope.loop_variable(loopNum, 1);
scope.input_from(&mut input).exchange(|x| *x)
.map(|x| work(&x, opVec[0]))
.concat(&cycle)
.map(|x| work(&x, opVec[2])) // loop body
.connect_loop(handle);
cycle.map(|x| (hash(&x), x)).probe(); // last line
});
Hello, in the code above I am trying to execute the loop body (commented above) n times using a loop then after I am done with the calculations I want to execute the last line. What is happening when I run it is that it executes the last line with the loop too, so both, loop body and last line, are looped. How can I fix this in such a way that after the whole loop is completed, I take the stream and I continue working on it?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the loop_variable, connect_loop, and probe calls shown in the Rust snippet, then trace how the loop output is scheduled. The issue is done when the loop body runs the requested number of times and the final stream processing runs only after the loop completes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100