Pometry / Pometry/Raphtory

`ShuffleComputeState::copy_over_next_ss` does not copy over global state

Open
#2,648 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
642
Forks
70
Avg merge
2d 13h
Merged PRs (30d)
21

Description

Draft PR #2554 is currently failing because it computes total_personalization once in init_steps. Then, in subsequent steps, it global_reads that sum. However, when ss is odd, and only when ss is odd, the expect panics. To illustrate, right before the failing read, I observed in LLDB that this was the state of total_personalization's index in the global state:

VecArray<double> {
  odd = size=0
  even = size=1 {
    [0] = 1
  }
  zero = 0
}

So the recorded total is still present, but at ss == 1 we try to read from odd, which is empty.
The "culprit" seems to be this line of code, which copies node-local states into the next superstep, but does not copy global into the next superstep. I quoted "culprit" because obviously this is rather complex machinery and I definitely want to leave room for the possibility that I just don't understand why this is intended behavior.

Assuming this is a bug, I tried to figure out why it hadn't surfaced before. Indeed, I'm still not really sure. It seems that the exact pattern of "read this global_agg every step" hasn't quite occurred, except in HITS, where it occurs with a counter that is also made into a global_agg_reset, which is not affected by this behavior. Moreover, finalize can call global_read e.g., in temporal_three_node_motifs, but the number of supersteps in that case is (as far as I can tell?) always the same.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in raphtory/src/core/state/shuffle_state.rs around the copy_over_next_ss implementation and inspect how global state is indexed across odd and even supersteps. Reproduce the failure described in Draft PR #2554, then compare global_read behavior with the existing HITS and temporal_three_node_motifs cases. Done means the odd-superstep read no longer panics and coverage verifies repeated global reads.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.