TimelyDataflow / TimelyDataflow/timely-dataflow
Allow making conditional regions
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 293
- Avg merge
- 14h 46m
- Merged PRs (30d)
- 4
Description
Scope::region() and Scope::region_named() have intrinsic overhead since in order to report events about all sub-operators it has to observe the traffic that they send and receive. In order to combat this, timely should offer some sort of optional region that takes a boolean which determines whether or not the region acts like Scope::region()/Scope::region_named() or whether it does nothing and is a noop. Regions are incredibly helpful for debugging and visualizing dataflows, but in large codebases they can end up having non-trivial overhead.
Example Code
Since writing something like this is not only painful for developers (and not always possible because of the dreaded generics) but also hurts both code reuse and compile times
if regions_are_enabled {
scope.region(|region| {
// ...inner dataflow graph
})
} else {
// ...inner dataflow graph
}
Optional regions would allow writing this instead
scope.optional_region(regions_are_enabled, |region| {
// ...inner dataflow graph
})
As per usual the normal Scope::optional_region() and Scope::optional_region_named() (or maybe Scope::optional_named_region()?) should be offered so that users can name the regions
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the implementations and callers of Scope::region() and Scope::region_named() in the Rust source, then trace how regions observe sub-operator traffic. Define the optional named and unnamed API behavior and verify that disabled regions are no-ops while enabled regions retain existing behavior; the issue does not name specific files or tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100