erigontech / erigontech/erigon

`recalcVisibleFiles`: make it self-contained, move more checks inside

Open
#13,174 0 comments 0 reactions 0 assignees View on GitHub
ErigonDB
Dominant language
Go
Stars
3.6k
Forks
1.5k
Avg merge
1d 16h
Merged PRs (30d)
455

Description

Now it depends on dirty files: `a.recalcVisibleFiles(a.DirtyFilesEndTxNumMinimax())`

Definition: "dirty files" - it's list where ok to have un-indexed files or files which exist only in 1 domain, etc... "visible files" - must be clean, with integrity-check between domains, same high, etc...

ToDo:
- at startup we must apply less checks - allow open files even if not all domains are exists (`kill -9` during build or merge).
- but `recalcVisibleFiles` must be more strict: check that domains are at same high, and all exist, etc... Example: `integrityCheck` func must be used not at `scanDirtyFiles` but at at `recalcVisibleFiles`.
- add couple unit-tests for above cases

`dirtyFilesEndTxNumMinimax` has business-logic:
```
func (a *Aggregator) dirtyFilesEndTxNumMinimax() uint64 {
m := min(
a.d[kv.AccountsDomain].dirtyFilesEndTxNumMinimax(),
a.d[kv.StorageDomain].dirtyFilesEndTxNumMinimax(),
a.d[kv.CodeDomain].dirtyFilesEndTxNumMinimax(),
// a.d[kv.CommitmentDomain].dirtyFilesEndTxNumMinimax(),
)
// TODO(awskii) have two different functions including commitment/without it
// Usually its skipped because commitment either have MaxUint64 due to no history or equal to other domains

//log.Warn("dirtyFilesEndTxNumMinimax", "min", m,
// "acc", a.d[kv.AccountsDomain].dirtyFilesEndTxNumMinimax(),
// "sto", a.d[kv.StorageDomain].dirtyFilesEndTxNumMinimax(),
// "cod", a.d[kv.CodeDomain].dirtyFilesEndTxNumMinimax(),
// "com", a.d[kv.CommitmentDomain].dirtyFilesEndTxNumMinimax(),
//)
return m
}
```
Let's move this business logic inside "integrity checkers" funcs.

In future will move schema out of `agg`.

I guess this task is required for future `Appendable` type (because they will likely create another `entities group` - with own integrites/prune_limits/etc...)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.