erigontech / erigontech/erigon
db: block files: long lifetime of `View`
- Dominant language
- Go
- Stars
- 3.6k
- Forks
- 1.5k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 455
Description
Technically `View` - it's ReadOnlyTransaction (analog from DBMS's) - and we need: high isolation level - which guarantee reads consistency for long time (between several `blockReader.GetBlockByNumber` calls).
so, we need "snapshot-isolation isolation level" :-)
https://en.wikipedia.org/wiki/Isolation_(database_systems)
https://en.wikipedia.org/wiki/Snapshot_isolation
E3 already providing this functionality on state files. See `Aggregator.BeginFilesRo + Aggregator.Close` (each time RPCDaemon does `tx := db.Begin` under the hood it cals `Aggregator.BeginFilesRo`.
Corner-cases to consider:
- erigon may start on empty datadir - and it will download files
- erigon may restart in the middle of download - means it will start on full datadir but will continue download
- erigon may restart in the middle of BuildMissedIndices - means it will start on "complete download" dir but not enough .idx files yet
- erigon may restart after download and buildMissedIndices done - but with --no-downloader flag - means it may not know "is download finish?"
- rpcdaemon may start without erigon (at all) on existing datadir - and it must immediately start serving RPC requests (it can't be blocked by request to erigon about "did you finish downloading?"). Now we solved this problem by storing in db list of "good files" - but it's not the best thing.
See also: https://github.com/erigontech/erigon/pull/11318#discussion_r1699278257
Sub-tasks:
- [x] https://github.com/erigontech/erigon/issues/11417
- [ ] Add `RoSnapshots` and `View` inside `temporal.DB/Tx`. BlockReader - to use this `View` instead of creating new one. Benchmark performance of RPC methods.
- [ ] Optional: lock-free `View` creation
Contributor guide
Assessment
This issue has not been assessed yet.