opensanctions / opensanctions/rigour
Process-lifetime symbol interner grows without bound on data-driven NUMERIC ids (yente query path)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 67
- Forks
- 18
- Avg merge
- 13h 7m
- Merged PRs (30d)
- 15
Description
The global symbol interner (rust/src/names/symbol.rs:77-106) never evicts, justified in-code by the id domain being "a bounded set (QIDs, org vocabulary, territory codes)". That premise doesn't hold: NUMERIC symbol ids are derived from arbitrary numeric tokens in analyzed names (rust/src/names/analyze.rs:354), and yente's match path runs analyze_names on every incoming query via nomenklatura logic_v2's entity_names.
Every distinct number ever seen in a query name is therefore retained for the process lifetime (~100 bytes each: Box<str> key + Arc<str> value). In a long-running HTTP server this is unbounded, attacker-influenceable memory growth — a client can grow the process indefinitely just by screening names containing fresh numbers.
Fix direction: unbounded-domain categories (NUMERIC at minimum) should bypass the interner, or the interner needs a size cap / eviction. The bounded-vocabulary categories (ORG_CLASS, territory codes, person-name QIDs) can keep the current behavior.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the global interner in rust/src/names/symbol.rs:77-106 and numeric-token handling in rust/src/names/analyze.rs:354, then trace how yente’s query path reaches analyze_names through nomenklatura logic_v2’s entity_names. Confirm that NUMERIC values no longer grow process-lifetime storage without bound while bounded-vocabulary categories retain their current behavior; add or run focused tests for repeated distinct numeric query names.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- performance, security
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100