ChainSafe / ChainSafe/lodestar

Error signing attestation - Corruption: bad entry in block

Open
#6,681 1 comment 0 reactions 0 assignees View on GitHub
meta-bug meta-investigate
Dominant language
TypeScript
Stars
1.4k
Forks
483
Avg merge
1d 18h
Merged PRs (30d)
156

Description

### Describe the bug

Got this errors all the time in a holesky node which cause all attestations missed
```
Apr-17 02:01:37.392[] error: Error signing attestation slot=1451408, index=16, head=0x907e5c902fb7bf0bd33614967aa327ebff1b1b29a660318e81cbba4e539bab75, validatorIndex=1535375 - Corruption: bad entry in block
Error: Corruption: bad entry in block
Apr-17 02:01:37.392[] error: Error signing attestation slot=1451408, index=53, head=0x907e5c902fb7bf0bd33614967aa327ebff1b1b29a660318e81cbba4e539bab75, validatorIndex=1535410 - Corruption: bad entry in block
```

this is caused by https://github.com/google/leveldb/issues/704
also notice that the error is recorded in `vc_slashing_protection_attestation_errors_total` metric

### Expected behavior

- The error was gone when I make a fresh `validator-db` folder and restart `vc` again, so we should add this to troubleshooting part of lodestar documentation
- This error comes from leveldb and throw all the way to "signing attestation" flow which make it confusing. We should find a way to signal that's an error coming from leveldb, for example `Database error - Corruption: bad entry in block`. One way is to wrap with an error in `@lodestar/db` package for all apis

### Steps to reproduce

_No response_

### Additional context

_No response_

### Operating system

Linux

### Lodestar version or commit hash

v1.17.0

Contributor guide

Open the contributing guide

Research direction

Start with the @lodestar/db package and trace how its errors reach the signing attestation flow, then review the validator-db troubleshooting documentation and the vc_slashing_protection_attestation_errors_total metric. Done means the LevelDB corruption recovery guidance is documented and database errors are distinguishable from generic attestation-signing failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend, databases, documentation
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.