hiero-ledger / hiero-ledger/hiero-consensus-node

State snapshot folder may be partially deleted at node shutdown

Open
#26,535 0 comments 0 reactions 1 assignee Claimed by @imalygin View on GitHub
Platform Stability
Dominant language
Java
Stars
406
Forks
226
Avg merge
3d 4h
Merged PRs (30d)
210

Description

This issue was observed recently. Right after node restart after shutdown, the following error was observed in the logs right at startup:

```
2026-07-18 03:42:36.561 6 WARN STARTUP SavedStateMetadata: Signed state does not have a metadata file at data/saved/com.hedera.services.ServicesMain/4/123/3358593/stateMetadata.txt
2026-07-18 03:42:36.562 7 ERROR EXCEPTION SignedStateFilePath: Unable to read saved state metadata file 'data/saved/com.hedera.services.ServicesMain/4/123/3358593/stateMetadata.txt'
```

It looks like there used to be a folder for round `3358593`, but something was wrong with it. Besides this round, there were 5 more folders under `data/saved/com.hedera.services.ServicesMain/4/123`.

Before restart, when the node was shut down, the very last two lines in the logs were:

```
2026-07-18 03:41:15.617 900 INFO STATE_TO_DISK <> FileUtils: deleting directory data/saved/com.hedera.services.ServicesMain/4/123/3358593
2026-07-18 03:41:15.742 901 INFO STARTUP <> Log4jSetup: JVM is shutting down.
```

The chances are the process was shut down before FileUtils completed deleting the specified folder.

One option is to change the ERROR log level above to WARN, since it doesn't affect node startup process much. Even better would be to add some synchronization between deleting old snapshot folders and JVM shutdown (but not at cost of increasing total node shutdown time).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.