TimelyDataflow / TimelyDataflow/differential-dataflow

Extracting arrangement from `threshold_total`

Open
#269 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
3k
Forks
211
Avg merge
10h 42m
Merged PRs (30d)
34

Description

Hi @frankmcsherry,

You mentioned several times in the past that threshold and threshold_total operators internally construct arrangements that can in principle be extracted and reused just like any other DD arrangement. I would like to implement this idea and was hoping you could point me towards the correct API. At the moment I have the following code in DDlog:

    my_collection.threshold_total(|_, c| if c.is_zero() { 0 } else { 1 })
                .map(|k| (k, ())).arrange()

Is my understanding correct that the arrangement created by the arrange operator above is a duplicate of the one built by threshold_total internally? If so, what would be the best way to extract this internal arrangement?

Thanks!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the threshold and threshold_total operators and compare their internal arrangements with the arrangement produced by arrange in the shown DDlog chain. Determine the API for extracting and reusing the internal arrangement; done means the duplicate arrangement is avoided and the extracted arrangement can be reused as described.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.