TimelyDataflow / TimelyDataflow/timely-dataflow

Communication: coalesce counters

Open
#243 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
3.6k
Forks
293
Avg merge
14h 46m
Merged PRs (30d)
4

Description

In communication/src/allocator/counters.rs the two Pushers counters should be able to coalesce pushed counts, rather than introduce each count as it happens. This requires that all code paths call done() on their pushers (or push a None item), which does not appear to currently be the case.

My recollection is that input is the ergonomically annoying case, where a user currently expects an input type that they can interact with without needing to signal "done-ness". A bit more discipline here (and perhaps some code breaks) should provide more clarity about when an input's output streams should propagate a done() signal (perhaps when the operator is executed; perhaps when the user says, perhaps something else entirely).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in communication/src/allocator/counters.rs by reading the two Pushers counters and tracing the code paths that use them. Identify where pushers currently fail to call done() or push a None item, with particular attention to input handling. Done means pushed counts can coalesce and the project has a clear, consistent rule for propagating done signals through input output streams.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
distributed-systems
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.