opensanctions / opensanctions/rigour

Process-lifetime symbol interner grows without bound on data-driven NUMERIC ids (yente query path)

Open
#229 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
67
Forks
18
Avg merge
13h 7m
Merged PRs (30d)
15

Description

The global symbol interner (rust/src/names/symbol.rs:77-106) never evicts, justified in-code by the id domain being "a bounded set (QIDs, org vocabulary, territory codes)". That premise doesn't hold: NUMERIC symbol ids are derived from arbitrary numeric tokens in analyzed names (rust/src/names/analyze.rs:354), and yente's match path runs analyze_names on every incoming query via nomenklatura logic_v2's entity_names.

Every distinct number ever seen in a query name is therefore retained for the process lifetime (~100 bytes each: Box<str> key + Arc<str> value). In a long-running HTTP server this is unbounded, attacker-influenceable memory growth — a client can grow the process indefinitely just by screening names containing fresh numbers.

Fix direction: unbounded-domain categories (NUMERIC at minimum) should bypass the interner, or the interner needs a size cap / eviction. The bounded-vocabulary categories (ORG_CLASS, territory codes, person-name QIDs) can keep the current behavior.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the global interner in rust/src/names/symbol.rs:77-106 and numeric-token handling in rust/src/names/analyze.rs:354, then trace how yente’s query path reaches analyze_names through nomenklatura logic_v2’s entity_names. Confirm that NUMERIC values no longer grow process-lifetime storage without bound while bounded-vocabulary categories retain their current behavior; add or run focused tests for repeated distinct numeric query names.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
performance, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.