hiero-ledger / hiero-ledger/hiero-consensus-node

Race condition free PCES files in state

Open
#13,307 3 comments 0 reactions 1 assignee Claimed by @cody-littley View on GitHub
Platform
Dominant language
Java
Stars
406
Forks
226
Avg merge
3d 4h
Merged PRs (30d)
210

Description

# Problem

For testing purposes, we current copy PCES files into the state snapshot directory. Unfortunately, this feature was rushed and the implementation is prone to fail (race conditions). When this effort fails it does not harm production environments (we ignore these files in production), but it does cause grief in our testing pipelines.

# Algorithm

Create a new component called something like `PcesSnapshotBuffer`. We will stream all preconsensus events into this component. Internally, the buffer will keep the events in topological order, and will remove ancient events from the buffer. However, instead of advancing the ancient threshold when consensus advances, it will advance the ancient threshold every time the state file manager advances.

When the state file manager receives a state, it needs to decide if that state should be written to disk or not. If not, it will tell the `PcesSnapshotBuffer` to advance its ancient threshold. If it decides to write that state to disk, it will request a list of all non-ancient events (with respect to that state) that are also ancestors of the judges of that state (a judge is considered to be an ancestor to itself). It will take those events and write them into a special PCES file in the state snapshot directory.

# Implementation
![pcesSnapshotBuffer](https://github.com/hashgraph/hedera-services/assets/56973212/ae93cc22-3c31-405f-bc4a-3164750baf0f)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.