erigontech / erigontech/erigon
Increase merge limit of Blocks segments to 500K
- Dominant language
- Go
- Stars
- 3.6k
- Forks
- 1.5k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 455
Description
Now blocks and bor files are 100K limited. It's because E2 was not able to release small files on R2. But now we can - so, we can afford bigger merge limit.
It will simplify code: now some chains has 500K limit for old files and 100K for new. (some users
It will improve performance: less shards (in sharded data) - better performance (less bloom filters checks, etc...). For example lookup txn by hash will be faster. (I saw this bottleneck in BSC pprof of trace_transaction)
Need do same for bor files (they are aligned with blocks)
This is not urgent task - let's work on it after 3.1 release.
Probably we don't need any special files change - all snapshotters and user's code will just do +1 merge in background and release naturally.
Also maybe it's good chance to change how `ExistenceFilter` works in recsplit index. I don't know if It make sense to do this features together or not (maybe not).
Contributor guide
Assessment
This issue has not been assessed yet.