handshake-org / handshake-org/hsd

Wallet filter can be stored on disk/db to speed up startup time.

Open
#874 0 comments 0 reactions 0 assignees View on GitHub
wallet-db
Dominant language
JavaScript
Stars
2.1k
Forks
306
PR merge metrics
No merged PRs in 30d

Description

Wallet filter is always in memory. When wallet db starts, it loads all relevant information that it needs to track from the database: address hashes, outpoint hashes and name hashes. This is walletdb specific, not wallet one. This can grow in size quickly if the wallet is active/old enough. To speed up start up time of the wallet we could, after every batch write, also write filter to either disk or the database (this makes it easier to make sure there are no race issues). On load if the file exists, we load filter from the file instead of the looping through all WID mappings of above mentioned hashes.

Also worth investigating the number of items, after which having filter makes no more sense and returns high % of false positives. At certain point it's simpler to just grab the full block every time. For those settings wallet can just track the number of items filter indexes and use those stats to decide if the filter should be used.

Contributor guide

Open the contributing guide

Research direction

Start by tracing walletdb startup loading, WID mappings, and the batch-write path to understand where the filter is built and restored. Compare storing it on disk versus in the database, and determine when its false-positive rate makes full-block retrieval preferable; done means persisted filters load safely and the size-based decision is defined.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
database
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.