TimelyDataflow / TimelyDataflow/differential-dataflow

Add traits and operators for totally ordered timestamps

Open
#74 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
3k
Forks
211
Avg merge
10h 42m
Merged PRs (30d)
34

Description

The CountTotal trait in operators/count.rs is a substantial simplification of the logic for a counting operator when the timestamps are totally ordered. This both leads to performance improvement, but also clarity improvements: the implementation is plausibly readable.

We should add these implementations for other operators, and perhaps think about whether we can provide these operators using Rust's specialization feature (incoming?), or other type-based methods rather than asking the user to know about and invoke them (a fine stopgap, but it requires others writing generic code to provide two variants).

This is a tracking issue for several candidate implementations, as will stick around until we come up with a sane way of helping the user find the methods through types (perhaps specialization, perhaps an OrderedCollection type, not clear).

  • Implement a DistinctTotal trait and operators. Issue #75.
  • Implement a GroupTotal trait and operators. Issue #76.
  • Implement a JoinTotal trait and operators. Issue #77.
  • Provide uniform methods dispatching on total or partial orderedness.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with operators/count.rs and the existing CountTotal implementation, then review issues #76 and #77 for the remaining GroupTotal and JoinTotal candidates. Done means the remaining total-order operator implementations are addressed and a clear approach is chosen for uniform dispatch between total and partial ordering.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.