TimelyDataflow / TimelyDataflow/timely-dataflow
Regions have non-trivial overhead
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 293
- Avg merge
- 14h 46m
- Merged PRs (30d)
- 4
Description
Regions have non-trivial overhead and cost a lot more than they should. Ideally a nested region (not scope) should be near-free aside from the cost of doing logging, but using each of the three versions of this example shows dramatic performance differences between them
| Version 1 (2 regions) | Version 2 (1 region) | Version 3 (No regions) | |
|---|---|---|---|
| 30,000 iterations | 12s | 8s | 5s |
I know there's extra things going on because of the .enter() and .leave() calls as well as the region subgraph operators themselves, but it's still a significant difference that grows even more apparent on larger applications
use timely::dataflow::{
operators::{Enter, Exchange, Input, Inspect, Leave, Probe},
InputHandle, ProbeHandle, Scope,
};
fn main() {
timely::execute_from_args(std::env::args(), |worker| {
let index = worker.index();
let mut input = InputHandle::new();
let mut probe = ProbeHandle::new();
worker.dataflow(|scope| {
let data = scope.input_from(&mut input);
// Version 1
scope
.region(|inner| {
let data = data.enter(inner);
inner.region(|inner2| data.enter(inner2).leave()).leave()
})
.inspect(move |x| println!("worker {}:\thello {}", index, x))
.probe_with(&mut probe);
// Version 2
scope
.region(|inner| {
data.enter(inner).leave()
})
.inspect(move |x| println!("worker {}:\thello {}", index, x))
.probe_with(&mut probe);
// Version 3
data
.inspect(move |x| println!("worker {}:\thello {}", index, x))
.probe_with(&mut probe);
});
for round in 0..30000 {
if index == 0 {
input.send(round);
}
input.advance_to(round + 1);
while probe.less_than(input.time()) {
worker.step_or_park(None);
}
}
}).unwrap();
}
A potential solution I can see would be to make a truly specialized (and separate) version of Subgraph that doesn't do any of the progress or input/output management that Subgraph does apart from the absolute minimum to have logging stay intact
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the three-region comparison from the issue and measure the 30,000-iteration timings. Then read timely/src/progress/subgraph.rs, especially Subgraph and its progress and input/output management, to identify the overhead attributable to nested regions. Done means nested regions approach the no-region cost while preserving logging behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100