erigontech / erigontech/erigon

db: block files: long lifetime of `View`

Open
#11,270 0 comments 0 reactions 0 assignees View on GitHub
ErigonDB
Dominant language
Go
Stars
3.6k
Forks
1.5k
Avg merge
1d 16h
Merged PRs (30d)
455

Description

Technically `View` - it's ReadOnlyTransaction (analog from DBMS's) - and we need: high isolation level - which guarantee reads consistency for long time (between several `blockReader.GetBlockByNumber` calls).

so, we need "snapshot-isolation isolation level" :-)
https://en.wikipedia.org/wiki/Isolation_(database_systems)
https://en.wikipedia.org/wiki/Snapshot_isolation

E3 already providing this functionality on state files. See `Aggregator.BeginFilesRo + Aggregator.Close` (each time RPCDaemon does `tx := db.Begin` under the hood it cals `Aggregator.BeginFilesRo`.

Corner-cases to consider:
- erigon may start on empty datadir - and it will download files
- erigon may restart in the middle of download - means it will start on full datadir but will continue download
- erigon may restart in the middle of BuildMissedIndices - means it will start on "complete download" dir but not enough .idx files yet
- erigon may restart after download and buildMissedIndices done - but with --no-downloader flag - means it may not know "is download finish?"
- rpcdaemon may start without erigon (at all) on existing datadir - and it must immediately start serving RPC requests (it can't be blocked by request to erigon about "did you finish downloading?"). Now we solved this problem by storing in db list of "good files" - but it's not the best thing.

See also: https://github.com/erigontech/erigon/pull/11318#discussion_r1699278257

Sub-tasks:
- [x] https://github.com/erigontech/erigon/issues/11417
- [ ] Add `RoSnapshots` and `View` inside `temporal.DB/Tx`. BlockReader - to use this `View` instead of creating new one. Benchmark performance of RPC methods.
- [ ] Optional: lock-free `View` creation

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.