opensanctions / opensanctions/opensanctions

Adopt the new FtM Control schema (edges, analyzer, crawlers)

Open
#5,568 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
810
Forks
187
Avg merge
1d 7h
Merged PRs (30d)
133

Description

followthemoney is gaining a Control schema (opensanctions/followthemoney#362): a sibling of Ownership under Interest with a directed controllercontrolled edge, both ranged LegalEntity, plus controlType (voting, appointment, management, contractual, trust, nominee, state, indirect) and legalBasis. It is for one entity controlling another without necessarily owning it, and it removes the need to cast controlled Organizations as Company just to satisfy Ownership.asset's Asset range.

Once a followthemoney release with Control is pinned here, the following needs doing.

1. Infrastructure that hardcodes Ownership

  • zavod/zavod/integration/edges.py:59PROTECTED_PROPS has per-schema entries for Ownership, Directorship, UnknownLink, … Add "Control": ["controlType", "role"] so control edges with conflicting descriptors are not merged on endpoints+dates alone.
  • datasets/_analysis/ann_graph_topics/analyzer.py:
    • SANCTION_ADJACENCY_EDGES (line 93) — add "Control".
    • rule_sanction_control_descent (line 274) — currently adjacent.schema.name != "Ownership" or prop.reverse.name != "owner"; it should also descend Control edges via controller. The topic it emits is already called sanction.control.
    • rule_export_control_descent (line 312) — is_a("Ownership") check; decide whether control edges should propagate export.control.linked too (the docstring warns against adding Directorship; Control is a different, deliberate assertion of control by the source).
    • Extend test_ann_graph_topics.py accordingly.

2. Crawlers that currently force control into other schemata

Ordered roughly by how clearly the source distinguishes control from ownership:

  • zavod/zavod/shed/bods.py:110 — BODS ownershipOrControlStatement interests are flattened into Ownership with interests[].type in role. BODS interest types votingRights, appointmentOfBoard, influenceOrControl, seniorManagingOfficial should become Control (with controlType mapped), while shareholding stays Ownership. Affects datasets/dk/cvr.
  • datasets/gb/coh/psc_parse.py — every PSC nature of control becomes a single Ownership; voting-rights ranges are parsed into percentage indistinguishably from share percentages. Split: ownership-of-shares-*Ownership; voting-rights-*, right-to-appoint-and-remove-*, significant-influence-or-control*Control. Voting-rights percentages will need a property on Control — raise that on followthemoney when this migration starts.
  • datasets/us/ofac/us_ofac_sdn.yml lookups.relations — "Owned or Controlled By" (15003) and "Owns, controls, or operates" (92019) map to Ownership. Consider Control for these (OFAC's own wording is disjunctive), keeping "Property in the interest of" (92122) as Ownership. Consumed by datasets/us/ofac/ofac_advanced.py.
  • datasets/lv/fiu_sanctions/crawler.py:160-181 — a list of "entities owned or controlled by" sanctioned persons, modelled as UnknownLink with the freeze type in role.
  • datasets/lt/fiu_freezes/crawler.py:74-77 — "property legally owned or controlled by a sanctioned entity", role-less UnknownLink.
  • datasets/_global/thesentry_atlas/crawler.py:20-41 — a literal CONTROLS relation is mapped to Directorship.
  • datasets/sg/mas_investor_alert/crawler.py:40-52,105-123 — "controlled by", "operated by", "managed by", "run by" all go through emit_ownership() into a role-less Ownership.
  • datasets/ee/ariregister/crawler.py parse_bfo() — beneficial owners with kontrolli_teostamise_viis_tekstina ("manner of exercising control") in Ownership.role; the control-based manners should become Control.
  • datasets/ua/war_sanctions/crawler_zyte.py — "Commercial ship manager" / "Ship Safety Management Manager" are UnknownLink; these are operational control of the owning entity's vessel and may fit Control only if the controlled party is a LegalEntity — check before migrating.

3. Remove the Company cast workaround

Where crawlers upgrade an Organization to Company solely so it can be an Ownership.asset, switch the relationship to Control where the source is asserting control, and drop the cast:

  • datasets/gb/fcdo_sanctions/crawler.py:352-356 (comment: "Ownership:asset must reference an Asset, which Organization isn't")
  • datasets/_global/gem_energy_ownership/crawler.py:95-98
  • Others found with the same pattern: us/hhs_exclusions, _global/gleif, sa/pcct_terrorism_list, br/* — audit each; some are genuine ownership of a company and should stay as they are.

Not affected: nomenklatura, zavod/exporters/senzing.py, consolidate.py, runner/util.py — all use schema.edge generically and pick up Control for free.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Pin a followthemoney release containing Control, then inspect zavod/zavod/integration/edges.py, datasets/_analysis/ann_graph_topics/analyzer.py, and the named crawler entry points. Start with test_ann_graph_topics.py and the crawler tests to establish current Ownership behavior. Done means Control is handled in infrastructure and applicable crawlers, with ambiguous mappings resolved and regression tests updated.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, data-engineering, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.