`ShuffleComputeState::copy_over_next_ss` does not copy over global state
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 642
- Forks
- 70
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 21
Description
Draft PR #2554 is currently failing because it computes total_personalization once in init_steps. Then, in subsequent steps, it global_reads that sum. However, when ss is odd, and only when ss is odd, the expect panics. To illustrate, right before the failing read, I observed in LLDB that this was the state of total_personalization's index in the global state:
VecArray<double> {
odd = size=0
even = size=1 {
[0] = 1
}
zero = 0
}
So the recorded total is still present, but at ss == 1 we try to read from odd, which is empty.
The "culprit" seems to be this line of code, which copies node-local states into the next superstep, but does not copy global into the next superstep. I quoted "culprit" because obviously this is rather complex machinery and I definitely want to leave room for the possibility that I just don't understand why this is intended behavior.
Assuming this is a bug, I tried to figure out why it hadn't surfaced before. Indeed, I'm still not really sure. It seems that the exact pattern of "read this global_agg every step" hasn't quite occurred, except in HITS, where it occurs with a counter that is also made into a global_agg_reset, which is not affected by this behavior. Moreover, finalize can call global_read e.g., in temporal_three_node_motifs, but the number of supersteps in that case is (as far as I can tell?) always the same.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in raphtory/src/core/state/shuffle_state.rs around the copy_over_next_ss implementation and inspect how global state is indexed across odd and even supersteps. Reproduce the failure described in Draft PR #2554, then compare global_read behavior with the existing HITS and temporal_three_node_motifs cases. Done means the odd-superstep read no longer panics and coverage verifies repeated global reads.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100