TimelyDataflow / TimelyDataflow/differential-dataflow
Add traits and operators for totally ordered timestamps
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 3k
- Forks
- 211
- Avg merge
- 10h 42m
- Merged PRs (30d)
- 34
Description
The CountTotal trait in operators/count.rs is a substantial simplification of the logic for a counting operator when the timestamps are totally ordered. This both leads to performance improvement, but also clarity improvements: the implementation is plausibly readable.
We should add these implementations for other operators, and perhaps think about whether we can provide these operators using Rust's specialization feature (incoming?), or other type-based methods rather than asking the user to know about and invoke them (a fine stopgap, but it requires others writing generic code to provide two variants).
This is a tracking issue for several candidate implementations, as will stick around until we come up with a sane way of helping the user find the methods through types (perhaps specialization, perhaps an OrderedCollection type, not clear).
- Implement a
DistinctTotaltrait and operators. Issue #75. - Implement a
GroupTotaltrait and operators. Issue #76. - Implement a
JoinTotaltrait and operators. Issue #77. - Provide uniform methods dispatching on total or partial orderedness.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with operators/count.rs and the existing CountTotal implementation, then review issues #76 and #77 for the remaining GroupTotal and JoinTotal candidates. Done means the remaining total-order operator implementations are addressed and a clear approach is chosen for uniform dispatch between total and partial ordering.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100