TimelyDataflow / TimelyDataflow/timely-dataflow
Communication: coalesce counters
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 293
- Avg merge
- 14h 46m
- Merged PRs (30d)
- 4
Description
In communication/src/allocator/counters.rs the two Pushers counters should be able to coalesce pushed counts, rather than introduce each count as it happens. This requires that all code paths call done() on their pushers (or push a None item), which does not appear to currently be the case.
My recollection is that input is the ergonomically annoying case, where a user currently expects an input type that they can interact with without needing to signal "done-ness". A bit more discipline here (and perhaps some code breaks) should provide more clarity about when an input's output streams should propagate a done() signal (perhaps when the operator is executed; perhaps when the user says, perhaps something else entirely).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in communication/src/allocator/counters.rs by reading the two Pushers counters and tracing the code paths that use them. Identify where pushers currently fail to call done() or push a None item, with particular attention to input handling. Done means pushed counts can coalesce and the project has a clear, consistent rule for propagating done signals through input output streams.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- distributed-systems
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100