erigontech / erigontech/erigon
compress: better handle `dictBuilder overflow` case
Open
ErigonDB
- Dominant language
- Go
- Stars
- 3.6k
- Forks
- 1.5k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 455
Description
If we producing many good pattren-candidates: they will come to dict-builder.
And dict-builder will sort them `first by score then alphabetically` (and evict tail).
it means high chance to see in dictBuilder `00` and `001` words (begin of alphabet) and don't see `ff` (end of alphabet).
Need: or better filter candidates before dictBuilder, or improve dictBuilder sorting logic, or do any other tricks. Or add one more limit: "soft limit of dict builder", "hard limit of dict builder", "hard limit of final dictionary".
Contributor guide
Assessment
This issue has not been assessed yet.