TimelyDataflow / TimelyDataflow/differential-dataflow
`wrapper/frontier.rs` does not update metadata
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 3k
- Forks
- 211
- Avg merge
- 10h 42m
- Merged PRs (30d)
- 34
Description
The TraceFrontier trace wrapper is used to forcibly advance timestamps in a trace, to bring data up to some common frontier (often because compaction may have occurred, and we want to ensure rather than hope that data have been advanced). However, the wrapper does not update the metadata about valid times, and in particular just mirrors the trace's advance_frontier and each batch's since frontier. Both of these should probably be lower-bounded by the frontier used to construct the wrapper.
I believe most operators are agnostic to these frontiers, and so this is probably not the source of any bugs of that flavor. However, it seems possible that operators should be less agnostic to them in the future (or now) and having incorrect metadata is never good.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in wrapper/frontier.rs with TraceFrontier, then trace how advance_frontier and each batch's since frontier are mirrored. Compare those metadata values with the frontier used to construct the wrapper, and update the behavior so the metadata reflects that lower bound. Verify the wrapper's frontier and batch metadata remain consistent after advancement.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100