erigontech / erigontech/erigon

compress: better handle `dictBuilder overflow` case

Open
#14,623 1 comment 0 reactions 0 assignees View on GitHub
ErigonDB
Dominant language
Go
Stars
3.6k
Forks
1.5k
Avg merge
1d 16h
Merged PRs (30d)
455

Description

If we producing many good pattren-candidates: they will come to dict-builder.
And dict-builder will sort them `first by score then alphabetically` (and evict tail).
it means high chance to see in dictBuilder `00` and `001` words (begin of alphabet) and don't see `ff` (end of alphabet).

Need: or better filter candidates before dictBuilder, or improve dictBuilder sorting logic, or do any other tricks. Or add one more limit: "soft limit of dict builder", "hard limit of dict builder", "hard limit of final dictionary".

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.