fireproof-storage / fireproof-storage/fireproof

[Data Integrity] Implement atomic commit with write-ahead logging (WAL)

Open
#1,499 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
973
Forks
58
PR merge metrics
No merged PRs in 30d

Description

## Summary

Database commits are not atomic - a crash during commit can leave the database in an inconsistent state with orphaned blocks. Implement a write-ahead log (WAL) for atomic commits with crash recovery.

## Problem

In `core/blockstore/commitor.ts`, blocks are written individually. If a crash occurs mid-commit:
- Some blocks may be written, others not
- The meta record may point to non-existent blocks
- Orphaned blocks consume storage without being reachable

## Proposed Solution

1. Design WAL entry format for commit operations
2. Create `WALService` at `core/blockstore/wal-service.ts`
3. Implement IndexedDB WAL storage
4. Implement two-phase commit protocol:
- Begin: Write WAL entry with commit intent
- Record: Write blocks with WAL reference
- Commit: Update meta record
- Finalize: Remove WAL entry
5. Implement rollback with orphan cleanup
6. Implement crash recovery on database open
7. Integrate WAL into commitor

## Expected Outcomes

- All blocks in commit succeed or fail together
- Partial failures trigger complete rollback
- Crash recovery restores consistent state
- Optional feature flag for WAL

## Dependencies

Should be implemented after the `validateBlocks` async fix (PR #1494).

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.