handshake-org / handshake-org/hsd
Wallet filter can be stored on disk/db to speed up startup time.
- Dominant language
- JavaScript
- Stars
- 2.1k
- Forks
- 306
- PR merge metrics
- No merged PRs in 30d
Description
Wallet filter is always in memory. When wallet db starts, it loads all relevant information that it needs to track from the database: address hashes, outpoint hashes and name hashes. This is walletdb specific, not wallet one. This can grow in size quickly if the wallet is active/old enough. To speed up start up time of the wallet we could, after every batch write, also write filter to either disk or the database (this makes it easier to make sure there are no race issues). On load if the file exists, we load filter from the file instead of the looping through all WID mappings of above mentioned hashes.
Also worth investigating the number of items, after which having filter makes no more sense and returns high % of false positives. At certain point it's simpler to just grab the full block every time. For those settings wallet can just track the number of items filter indexes and use those stats to decide if the filter should be used.
Contributor guide
Research direction
Start by tracing walletdb startup loading, WID mappings, and the batch-write path to understand where the filter is built and restored. Compare storing it on disk versus in the database, and determine when its false-positive rate makes full-block retrieval preferable; done means persisted filters load safely and the size-based decision is defined.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- database
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100