hiero-ledger / hiero-ledger/hiero-consensus-node

Test flake: signatures on PCES replay state

Open
#8,269 0 comments 0 reactions 0 assignees View on GitHub
Platform
Dominant language
Java
Stars
406
Forks
226
Avg merge
3d 4h
Merged PRs (30d)
210

Description

In a hard restart test, I observed a state written to disk without all of its signatures:

```
2023-08-29 16:12:02.991 358 ERROR EXCEPTION <> DefaultStateManagementComponent: state written to disk for round 136 did not have enough signatures. Collected signatures representing 22468825683/50000000000 weight. Total unsigned disk states so far: 1. {} [com.swirlds.logging.payloads.InsufficientSignaturesPayload]
```

Sequence of events:

- node reaches round 136, decides that it will eventually be written to disk
- node starts collecting signatures for round 136 but does not finish because we shut the node down without warning (intentionally)
- when the node comes back online, it regenerates round 136 during PCES replay
- since we don't create/send signature transactions for PCES rounds, this round is never signed
- the test logs an error when we write this state to disk without its signatures

Solution: for any test where we are doing a hard restart like this, we should ignore "InsufficientSignaturesPayload" messages in the log.

In the long term we will guarantee signing even if a round falls into a PCES replay time period. In the short term, this is not a high priority.

[swirlds.log](https://github.com/hashgraph/hedera-services/files/12468451/swirlds.log)

Contributor guide

Open the contributing guide

Research direction

Start by locating the hard-restart tests that perform PCES replay and inspect how log messages are asserted. Search for InsufficientSignaturesPayload and identify the affected test cases. Done means those tests ignore this expected payload during hard restart while still detecting other unexpected errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.