erigontech / erigontech/erigon
db: add drainReaders flag to BeginTemporalRw and TryBeginTemporalRo/TryView for MDBX GC
- Dominant language
- Go
- Stars
- 3.6k
- Forks
- 1.5k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 455
Description
## Summary
MDBX GC can only reclaim freed pages when `openTxs=1` at commit time (only the committing RW tx itself). Any RO tx opened **before** the RW tx holds an older snapshot, preventing the GC from advancing `cached_oldest` and reclaiming pages from prior transactions. This causes unbounded DB file growth.
RO txs opened **after** `BeginTemporalRw` see the same snapshot (txnid N) as the RW tx and do NOT block GC — they don't hold older snapshots.
## Proposed API Changes
### `BeginTemporalRw(ctx context.Context, drainReaders ...bool) (TemporalRwTx, error)`
When `drainReaders=true`:
1. Acquire exclusive write lock on a shared `sync.RWMutex` (the "commit gate")
2. Call the underlying `BeginRw` to create the MDBX write transaction
3. Release the write lock immediately
This drains all in-flight background RO txs that hold older snapshots before the RW tx is created. New RO txs can open immediately after — they see the same snapshot as the RW tx and won't block GC.
### `TryBeginTemporalRo(ctx context.Context) (TemporalRoTx, bool, error)`
Attempts `commitGate.TryRLock()`. If the gate is held by a committing writer, returns `(nil, false, nil)` instead of blocking. Callers can return cached/stale data.
### `TryView(ctx context.Context, f func(tx Tx) error) (bool, error)`
Same as `View` but uses `TryRLock`. Returns `(false, nil)` if a commit is in progress, allowing the caller to skip the read without blocking.
## Background
The commit gate is an `sync.RWMutex`:
- **Background readers** (collation `db.View()`, sentry status, etc.) hold `RLock` — multiple can run concurrently
- **Commit paths** set `drainReaders=true` on `BeginTemporalRw` — waits for existing `RLock` holders to finish, then creates the RW tx and releases immediately
- Lock hold time is **microseconds** (just the `BeginRw` call), not the full commit duration
Without this, background collation and sentry status `db.View()` calls open RO txs that overlap with commits, keeping `openTxs > 1` and preventing MDBX GC page reclamation — leading to unbounded DB growth (observed: 8 GB → 526 GB overnight).
## Implementation Notes
- The `sync.RWMutex` lives on the `Aggregator` and is exposed via `CommitGate() *sync.RWMutex`
- `TryBeginTemporalRo` / `TryView` are opt-in for background readers that can tolerate stale data
- Critical readers that MUST have fresh data continue using `BeginTemporalRo` / `View` (blocking `RLock`)
- The temporal DB layer (`kv_temporal.go`) implements the gate internally — callers don't manage locks directly
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Contributor guide
Assessment
This issue has not been assessed yet.