ChainSafe / ChainSafe/lodestar
Add retry mechanism for payload envelope DB persistence
- Dominant language
- TypeScript
- Stars
- 1.4k
- Forks
- 483
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 150
Description
## Context
In `persistPayloadEnvelopeInput()`, a DB write failure causes the in-memory `PayloadEnvelopeInput` to be pruned via `.finally()`, losing data from both DB and memory with no recovery path. This is the same pattern as `persistBlockInput()`.
### Current behavior
```typescript
await writePayloadEnvelopeInputToDb.call(this, payloadInput)
.catch((e) => {
this.logger.error("Error persisting payload envelope in hot db", {...}, e);
})
.finally(() => {
this.seenPayloadEnvelopeInput.prune(payloadInput.blockRootHex);
// ...
});
```
On a transient DB error (disk full, I/O timeout), `.catch()` logs the error but `.finally()` still prunes the in-memory data. The envelope is now gone from both DB and memory — irrecoverable.
## Proposed fix
1. Only prune on success (move prune from `.finally()` to `.then()`)
2. Add a retry mechanism for transient failures (exponential backoff, bounded retries)
3. Apply the same fix to `persistBlockInput()` which has the identical pattern
## References
- PR #8962 — `writePayloadEnvelopeInputToDb.ts` ([review thread](https://github.com/ChainSafe/lodestar/pull/8962#discussion_r2966245758))
- Same pattern exists in `persistBlockInput()` (noted by @twoeths)
/cc @nflaig @twoeths
Contributor guide
Assessment
This issue has not been assessed yet.