koala73 / koala73/worldmonitor

feat: CII v9 Country Instability Index — repair v8 in place, build v9 shadow service, cut over

Open
#7,582 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
86.6k
Forks
13.1k
Avg merge
8h 4m
Merged PRs (30d)
825

Description

> **Agent-native implementation plan.** Canonical file: `docs/plans/2026-09-02-001-feat-cii-v9-instability-index-plan.md` (`docs/plans/` is gitignored, so force-add it to share across worktrees). The plan exceeds the issue body limit, so it is split: this body carries the Goal Capsule, Product Contract, and Planning Contract; the four comments below carry Implementation Units U1–U8 (Track A), U9–U16, U17–U26, and System-Wide Impact through the Appendix. Reviewed 2026-09-03 by seven document-review personas plus seven maintainer rounds; all validated findings integrated. **Revision 8 — frozen.** v9.0 launches on three validated pillars (KTD-20); state disruption is published unvalidated and joins Level at v9.1 on real first-seen history; coups are scored leave-source-out; the monitoring floor is the named calibration output `MONITORING_FLOOR_CLUSTERS`; U26 phase A gates U23 and phase B gates U25.

## Goal Capsule

- **Objective.** Ship two tracks. Track A repairs the live CII v8 scorer so its conflict and unrest components stop reading zero, disclose what drives every score, and describe the methodology honestly. Track B builds CII v9 as a separate shadow service over 196 countries emitting Level, Momentum, Vulnerability, and Confidence with attribution, intervals, history, validation, and release gates, then cuts consumers over.
- **Authority hierarchy.** This plan > `AGENTS.md` and `CONTRIBUTING.md` > repo conventions cited in Sources. Where the plan and a test that pins current behavior disagree, the plan wins and the test is rewritten in the same unit (the pinned behavior is what is being repaired).
- **Stop conditions.** Stop and surface, do not guess, when: a source license blocks a core indicator (R14); the UCDP bulk download or GDELT export contract differs from Appendix A; an IDMC or UCDP credential is required and absent; a retrospective release gate in U20 fails after calibration; the investment gate after U11 fails; a prospective threshold breaches during the deprecation window (pause retirement, never retire on a breach); a cutover consumer in Appendix D cannot be migrated without a product decision.
- **Execution profile.** Track A units are sequential (U1 first, U5 after U3 and U4). Track B units U9 and U10 have no Track A dependency and can start immediately; U11 needs U3 and U4. Heavy checks run sequentially in a worktree, never in parallel (exit 137 risk).
- **Tail ownership.** Each unit lands as its own PR under the repo's tiered pre-push gate. Merge and auto-merge always require explicit human approval. Seeder units are not live until their Railway cron fires; backfill by hand after merge.

---

## Product Contract

### Summary

Repair CII v8 in place by wiring the GDELT event stream and UCDP fatalities into the scorer, defusing the dormant ACLED defects, and disclosing floors, boosts, and source states on the wire. Build CII v9 as a new `instability/v1` service that mirrors the Country Resilience Index engine's registry, imputation, confidence, interval, validation, and release-gate machinery, covers the 196-country universe, keeps durable daily history, and replaces v8 for all consumers only after its gates pass.

### Problem Frame

Production CII v8 publishes a score whose two heaviest live components are dark. ACLED has no credentials, so `fetchAcledCached` returns an empty array and every country's unrest and conflict components are zero; the outage boost is also dead because the producer writes `country` while the scorer reads `countryCode`. What remains is 40% editorial baseline plus news, military posture, hazard boosts, and floors, which is why five countries sit pinned at exactly 50 and Brazil ranks level with Israel. The public methodology still describes ACLED as a live input, and the chat analyst and brief builders read a payload shape that does not exist, so the in-app agents see no risk scores at all.

The repository already solved most of what v8 lacks. `seed-conflict-intel` and `seed-gdelt-bulk-materializer` parse the GDELT 15-minute export; the CRI engine has a typed indicator registry, four imputation classes, coverage-weighted confidence, weight-perturbation intervals, sensitivity and external-benchmark scripts, and committed release-gate fixtures over exactly the 196-country universe v9 needs. Neither is connected to CII.

An external audit of v8 (reviewed 2026-09-01) proposed the v9 shape adopted here: separate Level from Momentum, drop editorial multipliers and post-aggregation boosts, express observability as confidence rather than score inflation, and validate before ranking. This plan amends it where the review found it wrong: rank information today is destroyed by floors, not the ceiling; the conflict backbone must be UCDP + GDELT because ACLED is absent, not because of licensing alone; the state-disruption pillar has thin feeds and says so; momentum needs a cold-start rule and a magnitude floor; and the 196-country scale-out is an attribution problem, not a loop-bounds change.

### Requirements

**Track A — repair v8**

- R1. The v8 scorer consumes GDELT CAMEO daily aggregates for unrest and conflict counts and UCDP deaths for fatalities, so no component reads zero while those feeds are healthy.
- R2. Every score-affecting v8 change ships as formula version `v8.1`, with cache-key family, protocol hash, methodology docs in both locales, changelog entries, and public-surface literals updated in the same change.
- R3. Component formulas are monotonic: a higher event multiplier or a higher event count never lowers a component, and an absent feed is never scored as a verified zero.
- R4. Every `CiiScore` discloses source families with typed feed states, the floor applied and how far it displaced the blended score, and per-term contributions that reconcile exactly to the published score.
- R5. Dead readers are repaired: outage country attribution, chat-analyst and brief risk context, the `get_conflict_events` output schema, and the `check-country-risk` skill field names.
- R6. Health reports "ACLED not configured" as ok rather than as empty data, and content-age freshness over multiple sources reduces with min.

**Track B — construct**

- R7. v9 scores every country in the CRI rankable universe (196) and emits Level, Momentum, Vulnerability, and Confidence per country in the wire contract of KTD-12.
- R8. Level is a weighted mean of the validated pillars over observed slots with goalposts frozen per methodology version; no editorial baselines, per-country multipliers, post-aggregation boosts, or floors. `v9.0` validates and includes three pillars (armed conflict, political disorder, human impact); state-function disruption is computed and published with `status: 'unvalidated'` outside Level and enters Level at `v9.1` only after its indicators have 90 days of real first-seen history and pass the same gates (KTD-20).
- R9. Every indicator resolves to a typed state; unmonitored and source-failed slots contribute no fabricated value, and coverage is measured against design weights.
- R10. Confidence and a p05–p95 interval accompany every Level, computed in the same generation; when the draws fail the row carries `levelInterval: null` with `intervalStatus: 'unavailable'` rather than omitting the field. `rankEligible` is a per-row flag with `rankIneligibleReason`, and ineligible countries are visible but unranked.
- R11. Momentum measures each country against its own history with a magnitude floor and reports Insufficient History as an explicit state.
- R12. Vulnerability equals 100 minus the CRI structural-readiness pillar, carries the pillar's coverage, and reports `low-coverage` (null value) below 0.5 coverage and `unavailable` when the pillar is absent.
- R13. Exactly these overlay families are exposed beside Level and never enter it: hazards (climate anomalies, wildfires, earthquakes), travel advisory level and provenance, sanctions entry count, military posture (flights, vessels, GPS interference, AIS disruption), cyber incidents, and news pressure. Each carries `{ value, state, sourceKey, observedAt }` with the same typed states as indicators (`observed`, `stable-absence`, `unmonitored`, `source-failure`, `not-applicable`), so a null value is never rendered as "nothing observed" when the truth is "not monitored".

**Track B — data and seeders**

- R14. Core conflict and disorder inputs come from openly licensed sources (GDELT, UCDP CC BY 4.0, WorldMonitor's own feeds); ACLED is an optional adapter gated on credentials and license review.
- R15. GDELT ingestion covers QuadClass 3 and 4 for roots 14/17/18/19/20 from both the English and translation streams, deduplicates re-emitted events by signature, applies a mention floor, maps FIPS to ISO2 for the whole universe, and is idempotent per export file.
- R16. UCDP ingestion unions the annual GED file with every candidate monthly release from the open bulk download, deduplicating by `id` with the annual row winning.

**Track B — history, validation, cutover**

- R17. Daily history per country is retained at least 730 days, formula-tagged, and never spliced across methodology versions; monthly frozen snapshots carry methodology version, scorer commit SHA, retrieval cutoff, and a source manifest.
- R18. v9 passes the release gates in U20 before any consumer switches: sensitivity bounds, external benchmark, event-family backtest, monotonicity, no double counting, contribution reconciliation, coverage floors, and license audit.
- R19. Every v9 RPC has an MCP tool in the same release; responses carry methodology, data, and schema versions plus `computedAt`; field names are JMESPath-safe identifiers and country collections are arrays with `countryCode`.
- R20. Methodology docs in both locales are bound to the registry by tests, and a methodology RPC exposes the live registry.
- R21. v8 and `get_country_risk` keep v8 semantics until cutover; cutover migrates every consumer in Appendix D in one release, adds an `instability` block beside the `cii` block on `get_country_risk` rather than mutating it, and keeps the v8 endpoint and the `cii` block warm and reachable for a dated 90-day deprecation window announced in the changelog, the SDK READMEs, the docs banner, and `Deprecation`/`Sunset` response headers on the v8 route — "one release cycle" is not a unit external consumers can observe. Retirement at the end of the window is gated by the prospective validation (U26), pauses on a breach, and is extended when the cohort is short; the cutover flag stays reversible until retirement.

### Actors

- A1. Dashboard user reading the CII panel, map choropleth, country deep dive, and strategic-risk dial.
- A2. API and MCP agent consumers (`get_country_risk`, `get_conflict_events`, analysis tools, the chat analyst, brief generation, external SDK users).
- A3. Operator running seeders on Railway, watching `/api/health`, and cutting over.
- A4. Methodology reader (researcher, journalist, search engine) citing the docs page or a frozen snapshot.

### Key Flows

- F1. Scorer tick (Track A)
- **Trigger:** `getRiskScores` cache miss or relay warm-ping.
- **Steps:** read GDELT daily aggregates, UCDP events, auxiliary keys with typed Read Outcomes; compute components; apply floors; emit disclosure; persist live, stale, trend snapshot, seed-meta.
- **Outcome:** `CiiScore[]` with `methodologyVersion: 'v8.1'`, contributions reconciling to `preFloorScore`.
- F2. v9 scoring tick
- **Trigger:** `seed-instability-scores` cron (15 min) calls the refresh RPC.
- **Steps:** resolve every indicator per country to a typed state; score pillars; compute Level, Confidence, Vulnerability, and intervals; read history for Momentum; write generation-suffixed per-country and interval records, then the ranking as the single atomic flip; write seed-meta, the refresh audit, and the shadow comparison; write the daily history member and raw record once per UTC day.
- **Outcome:** one complete generation published for 196 countries; health green.
- F3. Cutover
- **Trigger:** U20 retrospective gates pass, Shadow Measurement holds per KTD-17 (14 days spanning a UCDP release day and a CRI republish, each proven processed), every v8 Tier-1 country is rank-eligible in v9 except the bounded closed-media allowlist in Success Criteria, and at least 80 countries are rank-eligible. The prospective layer is not a trigger.
- **Steps:** enable gateway, flip dashboard flag, add the `instability` block to `get_country_risk` beside its `cii` block, update public surfaces and drift tests, migrate Appendix D consumers; keep v8 warm and the cutover flag reversible.
- **Outcome:** all consumers read v9; v8 endpoint and the `cii` block still answer through the dated 90-day deprecation window while the prospective cohort accrues.
- F4. v8 retirement (U25)
- **Trigger:** end of the 90-day deprecation window with U26 reporting `pass` (cohort at or above the pre-registered minimum and every prospective bound met, Success Criteria).
- **Steps:** confirm the monthly prospective report; switch the v8 endpoint to its retired response and leave the deprecated `cii` field and the v8 RPC declared (the buf FILE breaking policy forbids deleting them until a v2 API); remove keys, seed-meta, health rows, and the warm-ping together; retire the banner; close the rollback flag.
- **Outcome:** v9 is the only CII (terminal state `Retired`). A threshold breach pauses retirement and opens the rollback review: the CII owner (the maintainer who approves methodology bumps) decides within five working days of the breach report; a breach on precision at 10 or the false-alert rate rolls back (cutover flag off, v8 primary, v9 back to Gated), a breach on lead or cooling accuracy only pauses retirement and extends the dual run; the decision and its evidence are appended to the shadow artifact. An unmet cohort extends the dual run without a review.

### Acceptance Examples

- AE1. **Given** GDELT daily aggregates show 12 protest events (mentions ≥ 2) for France in the last 7 days and 0 for Syria, **when** the v8.1 scorer runs, **then** France's unrest component exceeds Syria's, and lowering France's multiplier never raises it. Covers R1, R3.
- AE2. **Given** ACLED credentials are unset, **when** `/api/health` evaluates the dedicated `acledSource` row, **then** it reports `NOT_CONFIGURED` bucketed ok, the `riskScores` row is unaffected, and the score's `dataCoverage.acled` reads `unconfigured`. Covers R4, R6.
- AE3. **Given** Lebanon's blended v8.1 score is 41 and its advisory floor is 50, **when** the score is published, **then** `combinedScore` is 50, `floorApplied.kind` is `advisory-reconsider`, and `floorApplied.displacement` is 9. Covers R4.
- AE4. **Given** a country whose GKG monitoring volume over 30 days (distinct publisher clusters locating it) is below the floor, **when** v9 scores it, **then** its GDELT indicators carry class `unmonitored` with `monitoringVolume` disclosed, its armed-conflict pillar coverage falls accordingly, and `rankEligible` is false with `rankIneligibleReason: 'low-coverage'` if weighted coverage drops below 0.70. Covers R9, R10.
- AE5. **Given** a country with 10 days of v9 history, **when** Momentum is computed, **then** `momentumStatus` is `insufficient-history`, `momentum` is null, and `momentumLabel` is `warming-up`; **given** 40 days, **then** `momentumStatus` is `provisional` and momentum is computed against the pooled cross-country MAD; **given** 120 days, **then** `momentumStatus` is `ok` and momentum uses the country's own MAD-floored distribution. Covers R11.
- AE6. **Given** methodology version changes from `v9.0` to `v9.1`, **when** a 30-day history is read, **then** only `v9.1` members are returned, the response carries `versionBreaks: [{date, from: 'v9.0', to: 'v9.1'}]` as boundary metadata, and Momentum uses `v9.1` members only. Covers R17.
- AE7. **Given** an agent calls `get_instability_ranking` with defaults (`rankEligibleOnly=true`, `limit=50`), **when** 60 countries are rank-eligible, **then** the top 50 by rank are returned with `total: 60` and a continuation, `limit=100` returns all 60, and `rankEligibleOnly=false` returns ineligible rows with `rankEligible: false` and their reason. Covers R10, R19.
- AE8. **Given** the cutover flag is on and a fixture holds one rank-eligible country, one ineligible country, and one country still warming up, **when** the dashboard renders, **then** the CII panel shows Level, momentum label, and confidence for the first, an explicitly marked unranked row with a legend key for the second, and a `warming-up` momentum label for the third, and the panel's methodology link opens the v9 methodology page. Covers R10, R11, R21.

### Success Criteria

- No Tier-1 country publishes `unrest = 0 and conflict = 0` on a day when GDELT recorded material-conflict or protest events for it.
- Track A counterfactual matrix shows the per-country delta attributable to wiring versus baseline deflation; the artifact is committed.
- v9 retrospective release gates in U20 pass: Spearman between the 12-month mean of Level and WGI Political Stability ≤ −0.55 over the comparable-denominator cohort (rows with `levelStatus: 'ok'` and every validated pillar at coverage ≥ 0.5; WGI scores stability, so the expected correlation is negative; ties mid-ranked; the all-finite and same-month figures reported ungated) and ≤ −0.50 over that cohort's non-Tier-1 members, and the time-split backtest beats both the raw-event-count baseline and repaired v8.1 over Tier-1 with a strictly positive clustered-bootstrap lower bound on precision at 10, non-inferior lead, false-alert upper bound ≤ 0.30, and cooling lower bound ≥ 0.70.
- v8 is retired (U25) only when the prospective validation (U26) passes at the end of the 90-day deprecation window: superiority over both baselines on precision at 10 (strictly positive lower bound of the 90% country-clustered bootstrap interval), non-inferior lead, false-alert upper bound ≤ 0.30, cooling lower bound ≥ 0.70, on a cohort at or above the minimum set by the pre-registered power simulation; a breach pauses retirement and opens the rollback review defined in F4, and an unmet cohort extends the dual-run period.
- At cutover every v8 Tier-1 country is rank-eligible in v9, except at most five closed-media states whose GDELT status is `unmonitored` and which are listed in the shadow artifact with the unresolved slot named; a sixth ineligible Tier-1 country blocks cutover. Globally, at least 80 countries are rank-eligible on cutover day (the same bar as the investment gate), so the map does not launch with only the Tier-1 cohort usable.
- After cutover, every consumer in Appendix D reads v9 and the public literal reads "scored for 196 countries, ranked for N", with N taken from the frozen snapshot's rank-eligible count; the negative regex tests forbidding "CII … 196 countries" are inverted to bind that phrasing.

### Scope Boundaries

- Rebuilding the CRI engine or coupling v9 to its code is out; v9 copies patterns and reuses only pure helpers.
- Forecast probabilities ("35% chance of escalation") are out; CII remains a measurement. Lead time and precision at 10 in the backtest validate the timeliness of the measurement, not a predictive claim; public copy and tool descriptions never use "predict", "forecast", or "warn" for momentum bands.
- CRI code stays untouched except one metadata edit: the UCDP license entries in `resilience/v1/_indicator-registry.ts` move from `research-only` to `open-attribution` (KTD-4), so the repo does not assert two licenses for one source.
- Point-in-polygon geocoding for coordinate-only overlay feeds is out; overlays keep the v8 bbox attribution for Tier-1 and ISO-code attribution elsewhere.
- Obtaining an ACLED license is a business decision outside this plan; the adapter seam is in scope.

#### Deferred to Follow-Up Work

- Internet-shutdown feed beyond Cloudflare Radar `GOVERNMENT_DIRECTED` outages (IODA or NetBlocks).
- Momentum band transitions feeding the cross-source signal stream (Regime Transition pattern) — after U18 stabilizes.
- Automated coup and emergency-measure detection from GDELT GKG themes replacing the curated `state-ruptures.json`.
- The `expanded_risk_scores` composite MCP tool already deferred in `api/mcp/registry/cache-tools.ts`.
- Issue #4276 composite country history (`history:country:{cc}`) consuming v9's history key.
- Fixing issue #5852 (display-key rolling window resets on source-tag flip); Track A avoids the key instead.

### Outstanding Questions

- Q1 (deferred, product — decide before U9 lands its entitlement entries). Access tier per v9 route and tool, resolved together with Q2 because the defaults are only coherent as a pair. Default: the compact ranking route `get-instability-scores` and the methodology route are public (the anonymous dashboard reads them, as it reads `get-risk-scores` today) and their MCP tools `get_instability_ranking` and `get_instability_methodology` are `free-account`; the full single-country record `get-instability-score` and `get-instability-history` are tier 1 in both entitlement registries with `subscription` MCP tools.
- Q2 (deferred, product, paired with Q1). Whether the incidental free-account exposure of the CII table through `get_conflict_events` survives cutover. Default: it carries v9 compact rank-eligible rows, which under the Q1 default gives away nothing the free ranking tool does not already serve.
- Q3 (deferred, data). Whether HDX HAPI's IDMC-sourced IDP series is granular enough for 30-day flows. Default: U12 uses HAPI (already configured) and the human-impact pillar runs on UCDP civilian deaths plus displacement stock delta wherever the flow series is absent.
- Q4 (deferred, product). Whether `/country-instability-index/` and the crawlable corpus move to v9 at cutover or one cycle later, and whether ineligible countries get indexed rows. Default: at cutover, inside U23's deploy → verify → freeze → literals sequence, with ineligible countries listed in a separate unranked table.
- Q5 (deferred, product). Whether an instability map that leaves closed states (KP, ER, TM) unranked and hatched meets the dashboard user's expectation, or whether those rows need an editorial note. Default: hatched with the text marker from AE8; no editorial score.
- Q6 (deferred, data). The license of HDX HAPI's IDMC-sourced IDP series feeding `hi_newDisplacement`; if it is non-commercial the U20 license audit fails on a core indicator. Checked in the U12 spike before any seeder work.

---

## Planning Contract

### Key Technical Decisions

- **KTD-1. Track A reads a purpose-built GDELT daily-aggregate key, not the display key.** `conflict:acled:v1:all:0:0` holds a 24-hour rolling window that resets whenever the seeder's Source Tag flips between `gdelt-bulk` and `gdelt` (issue #5852, open), and the v8 formula needs a 30-day window. `seed-gdelt-bulk-materializer` already fetches every export file, so it writes one more seed-owned key of per-country, per-UTC-day CAMEO class counts and mention sums with a per-export-file cursor. Both tracks read it. Rejected: reading `unrest:events:v1` (GKG themes, no CAMEO typing, unknown window) and adding pagination to the ACLED client (no credentials to paginate).
- **KTD-2. GDELT ingestion admits QuadClass 3 and 4 and maps CAMEO to scorer vocabulary explicitly.** Root 14 (protest) is QuadClass 3, so the existing `QuadClass === '4'` gate can never return a protest. Mapping: 14 → protests, 145 → riots, 19 → battles, 183 → explosions, other 18x and 20x → civilian violence, 17 (1723 curfew, 1724 emergency, 173 arrest, 175 repression) → state coercion. Events are deduplicated by signature `(SQLDATE, Actor1Code, Actor2Code, EventCode, ActionGeo_CountryCode, ActionGeo_ADM1Code)` only within 180 minutes of the first sighting by `DATEADDED` (the 8.7% re-emission was measured over a 90-minute horizon; a window-wide dedup would collapse distinct same-day events, especially where ADM1 is blank), carrying the maximum `NumMentions` and `NumSources` across collapsed rows; rows are kept only when `NumMentions ≥ 2`, and `NumSources` is carried as the corroboration weight. A per-country-day list of distinct GKG publisher clusters locating the country (`gkgClusters`; the raw registrable-domain list `gkgDomains` is kept only as a diagnostic) feeds the monitoring floor (KTD-11): domains are reduced to registrable form (eTLD+1) and then merged into syndication clusters — domains that publish near-identical titles (GKG `PAGE_TITLE` hash) for the same country within the same day are one cluster, and a committed `shared/publisher-clusters.json` folds known multi-domain operators — so ten domains under one operator count once; blank-actor rows are excluded from breadth and count at weight 0.5. A sustained-elevation rule covers gradual manipulation below the winsor cap: a country whose GDELT-derived intensity stays above twice its pre-elevation 90-day median for more than 14 consecutive days without non-GDELT corroboration is treated exactly like an anomaly-flagged one (rank withheld, reason `anomaly`) until corroborated. Manipulation guard: each country's daily class counts are winsorized at the p99 of its own 730-day history computed over non-flagged days only (pooled p99 during warm-up), so a campaign cannot raise its own cap; a rapid momentum band requires corroboration from at least two independent outlets on average (`NumSources ≥ 2` weighted) and from a non-GDELT family (UCDP, outages, or a curated rupture) within 30 days; and a country whose GDELT intensity jumps above the winsor cap without any non-GDELT corroboration carries `anomalyFlag: true`, has its momentum label capped at `rising`, and is withheld from ranking (`rankEligible: false`, `rankIneligibleReason: 'anomaly'`). The flag clears only on non-GDELT corroboration, never by elapsed time; after 14 uncorroborated days it is reported in health and the monthly report, so a sustained media campaign can move a published Level but cannot buy a rank. The flag is disclosed on the row and counted in the KTD-17 shadow gate. The translation stream is merged (99% additive; it carries the non-Anglophone countries). Incremental ingest partitions by `DATEADDED`; history rebuilds aggregate by `SQLDATE`.
- **KTD-3. FIPS→ISO2 comes from GeoNames `countryInfo.txt`, joined on code, with four manual rules.** GDELT geo codes are FIPS 10-4 and 71 of them are valid ISO codes for a different country (RS Russia/Serbia, CH China/Switzerland, NI Nigeria/Nicaragua, IS Israel/Iceland). Rules: `GZ`/`WE` → `PS`, `RB`/`YI` → `RS` (GDELT emits the undocumented `RB` for Serbia), `KV` → `XK`; drop `OS`, `UF`, `UU`, `NM`, `AY`. The table lives at `shared/fips-to-iso2.json` with a byte-identical `scripts/shared/` mirror per `tests/scripts-shared-mirror.test.mjs`, and a test pins every trap in Appendix B.
- **KTD-4. UCDP moves to the open bulk download and accumulates candidate releases.** The API now returns 401 without a token; the CSV zips need none. Candidate monthly files are incremental (each holds events coded that month regardless of event date; the latest file alone is ~16% of the year), so the seeder unions the annual file with every candidate file, supersedes by period (candidate rows whose `date_start` falls in a year the newest annual file covers are discarded, because events that fail annual inclusion are dropped rather than re-issued under their id), dedups by `id` within uncovered years with the latest candidate winning on revised `best`, and publishes a per-country monthly aggregate key for 36 months. `conflict:ucdp-events:v1` keeps its shape for existing readers. UCDP's downloads page states all datasets are CC BY 4.0; the CRI registry's `research-only` entries for UCDP are corrected to `open-attribution` in U4 so the license audit does not contradict the CRI linter.
- **KTD-5. Fatalities come from UCDP only; GDELT never fabricates deaths; monthly sources get monthly windows.** GDELT has no fatality field. v8.1's `fatalityScore` uses UCDP `deathsBest` over the trailing 90 days from the monthly aggregate key with no recency step, because a source that lands once a month with up to a month of coding lag cannot drive a 7-day weighting without a universe-wide sawtooth on release day. v9's armed-conflict pillar splits event intensity (GDELT, daily) from lethality and civilian targeting (UCDP, 90-day monthly window) so the two source families corroborate rather than double count, and UCDP-sourced indicators are excluded from the series momentum differentiates (KTD-13).
- **KTD-6. The unrest formula becomes one log-scaled curve for every country.** Replace the `multiplier < 0.7` branch with `logScaledScore(unrestCount × multiplier, cap, pivot)`, the same shape the conflict component already uses. Pure linear with the 50-point cap saturates at six to nine events and makes unrest binary. Pivots are set by `scripts/calibrate-cii-pivots.mjs` from the observed 30-day distribution across Tier-1 countries (p90 of the weighted count maps to 80% of the cap) and recorded in `_risk-config.ts` with the calibration date; the constant is a stated policy choice, not presented as forced.
- **KTD-7. Version `v8.1` for the repaired lineage; `v9.x` is reserved for the new index.** The protocol-hash test forces a bump on any literal change. `v8.1` avoids the collision with the new index and reads as "v8 repaired". Six files hardcode `risk:scores:sebuf:v8`; a drift test asserts all copies equal the derived key before the bump lands. The repo's tests assume integer versions, so U5 and U7 update every site that matches `v\d+`: `tests/cii-scoring.test.mts` ~2411–2413 (unescaped interpolation) and ~2476 (`sebuf…:v\d+` consumer regex), `tests/cii-docs-drift.test.mts` ~96 (`v\d+\s+`), `tests/crawlable-corpus.test.mjs` literals, and the `CII_PROTOCOL_SNAPSHOT_HASH_BY_VERSION` key (a quoted string key). Rejected: bumping to `v9`, which would put the label of the new index on the old scorer's wire.
- **KTD-8. Disclosure fields are additive on the wire and enforced on the serialized form.** New `CiiScore` fields: `sourceFamilies[]`, `dataCoverage{acled, gdelt, ucdp, outages, news, military}` with values `ok | stale | missing | unconfigured | failed`, `floorApplied{kind, floorValue, preFloorScore, displacement}`, `contributions[{term, points}]` (baseline, four weighted components, each boost) whose sum before floor equals `preFloorScore`. Tests assert on JSON output, not in-process objects, because `undefined` vanishes from JSON and `null` does not.
- **KTD-9. v9 is a separate service that copies CRI patterns and shares only pure helpers.** Every CRI change forces a 196-country score migration and namespace rotation across score, history, and interval keys; coupling would tie v9 iteration to CRI's release cadence. Copied: `IndicatorSpec` registry shape, four imputation classes plus `incomplete`, `weightedBlend` with `certaintyCoverage`, coverage against design weights, indicator trace for drivers, live/stale/seed-meta trio, two-layer cache isolation (key prefix plus in-payload formula tag rejected at read), release-gate fixtures. Shared by import: `_rankable-universe.ts`, `resilience-freshness.ts#classifyStaleness`, country resolvers, and the weighted-blend helper, which is extracted to `shared/weighted-blend.ts` with its own tests and imported by both engines — a pure function shared by import forces no cache-namespace rotation, whereas a byte-parity test against a private copy would have re-coupled the two release cadences.
- **KTD-10. Unresolved slots renormalize onto surviving slots; coverage tells the truth.** This is the repo's Component Slot convention. Withholding is never automatically cautious (a lost slot raises the score when survivors read higher), so every indicator failure is typed, coverage is measured against design weights, `rankEligible` gates on coverage, and the counterfactual matrix (type-corrupted fixtures, not merely absent) is run through the real scorer before calibration is frozen.
- **KTD-11. Missing-data semantics are inverted from CRI deliberately.** In CRI, stable absence imputes a high (good) score. In v9, stable absence from a comprehensive source (UCDP, UNHCR) is an observed zero with certainty 0.7; absence from an event-scraping source (GDELT) is `unmonitored` unless the country clears the monitoring floor, in which case zero conflict events is `observed`. The floor measures whether GDELT's sources cover the country at all, not whether anything happened there, so it is based on GKG coverage breadth — the number of distinct publisher clusters (KTD-2: registrable domains folded by syndication and known ownership) publishing articles that locate the country in 30 days, computed as the union of the daily `gkgClusters` lists the U3 aggregate carries (from the GKG stream the materializer already parses; a calm country still draws diplomatic, economic, and sports coverage), with the threshold being the named registry constant `MONITORING_FLOOR_CLUSTERS` — a calibration output of U24 (its pre-calibration default is 10 clusters and every other section refers to it by name, never by number), because raw article volume could be forged by one outlet while cluster breadth cannot without many independent publishers. Calibration selects the constant and the holdout alone gates it: U24 places `MONITORING_FLOOR_CLUSTERS` where GDELT's material-conflict detection rate for UCDP-recorded conflict stops falling across the calibration window, and U20 gates that detection rate on the post-2025-06-30 holdout only, never on the data that chose the constant. Attention still is not detection — without ACLED there is no ground truth for protest detection — so the floor's limits are stated in the methodology page. The row publishes `monitoringVolume` (the 30-day cluster count) so a reader can see why a country is unmonitored; raw domain counts are never used for the floor. `not-applicable` is emitted only where the construct cannot apply (aviation closures for a country with no curated airport; displacement stock delta where UNHCR lists the country as a host-only entry). `unmonitored`, `source-failure`, and `not-applicable` slots are excluded from the mean and reduce coverage; they are never imputed to 50, because 50 instability for an unmonitored country is a fabricated fact.
- **KTD-12. Wire contract.** `InstabilityScore { countryCode, countryName, level, levelStatus, levelInterval{p05,p95}, rankStable, momentum, momentumStatus, momentumLabel, vulnerability, vulnerabilityStatus, vulnerabilityCoverage, confidence, confidenceBreakdown{freshness, coverage, agreement, attribution}, weightedCoverage, rankEligible, rankIneligibleReason, rank, anomalyFlag, monitoringVolume, intervalStatus, intervalDraws, pillars[{id, score, weight, coverage, status}], drivers[{indicatorId, pillar, contribution, rawValue, normalized, sourceFamily, imputationClass, observedAt}], sourceStatus[{family, state, lastObservedAt, complete}], overlays{hazards, advisory, sanctions, militaryPosture, cyber, newsPressure}, methodologyVersion, dataVersion, schemaVersion, generationId, computedAt, comparability }` where `comparability` is the fixed non-comparability statement of KTD-19. `levelStatus` is `ok` or `insufficient-coverage` (level null when every pillar is insufficient); `vulnerabilityStatus` is `ok`, `low-coverage`, `stale`, or `unavailable`; `momentumStatus` is `insufficient-history`, `provisional`, `ok`, or `lost`; `sourceStatus` names source families, not Redis keys. Level display bands are registry constants derived from the calibration distribution and published with the methodology, not client-side literals. Snake case in proto, camel case on the wire as generated; every name is an unquoted JMESPath identifier; collections are arrays. Optional numerics use the house `*Available` companion pattern where proto3 needs it.
- **KTD-13. Momentum differentiates a 7-day activity series, not the 30-day Level, with two floors and explicit warm-up states.** Level's recency weighting would make Δ-of-Level emit "easing" eight days after every burst with no new information, so momentum uses `A_t` = the sum of the daily-sourced normalized event indicators (GDELT protest, coercion, and material-conflict intensities; outages) over the trailing 7 days, on the same 0–100 scale; UCDP-sourced and annual indicators are excluded (KTD-5). `Δ7 = A_t − A_{t−7}`; scale = MAD of the country's own Δ7 series over up to 730 days; `MADfloor = max(MAD, 1.5, 0.5 × global median MAD)`; `z = 0.6745 × Δ7 / MADfloor`; `momentum = round(100 × tanh(z / 2))`. Bands: `stable` if `|momentum| < 15`; `rising` or `easing` to 50; `rising-rapidly` or `easing-rapidly` beyond 50, and a rapid label also requires `|Δ7| ≥ 5` (percentile alone must never decide the band). States: under 14 days of history → `momentum: null`, `momentumStatus: 'insufficient-history'`, `momentumLabel: 'warming-up'`; 14–89 days → `provisional` with the pooled cross-country MAD; ≥ 90 → `ok` with the own distribution; a window that was previously ≥ 14 days and has receded → `lost` (label `history-lost`), distinguished by a `historyDepthReached` marker in seed-meta. A single-day burst must not produce an easing band on day 8 or day 31; U18 tests it.
- **KTD-14. Confidence = 35 freshness + 30 coverage + 20 agreement + 15 attribution, all in [0,1].** Freshness: weighted mean over indicators of 1 within cadence budget, decaying linearly to 0 at 3× budget, using `classifyStaleness`. Coverage: weighted coverage over core indicators. Agreement: for pillars with two source families, `1 − |sA − sB| / 100` on the two families' normalized slot scores for that country (cross-universe rank percentiles were rejected because ~130 zero-mass ties would compress the term to a constant); 0.5 when either family is unresolved or the pillar has one family. Attribution: weighted share of evidence attributed by structured ISO code (1.0), geocoded output including GDELT's FIPS-mapped `ActionGeo_CountryCode` (0.9), editorial record (0.8), name match (0.6). `rankEligible = weightedCoverage ≥ 0.70 ∧ every validated pillar's coverage ≥ 0.5 ∧ freshness ≥ 0.5 ∧ no armed-conflict source-failure ∧ no uncorroborated anomaly`; the per-pillar clause exists because Level renormalizes over surviving pillars, and ranking a country scored over three pillars against one scored over four would compare incomparable denominators — such rows keep their Level and `levelStatus: 'ok'` but are not ranked. Every ineligible row carries `rankIneligibleReason` (`low-coverage`, `pillar-coverage`, `stale-evidence`, `source-failure`, `anomaly`), the first failing clause in that order, so the UI marker and agents state the actual reason. The earlier `confidence ≥ 50` clause was a Vacuous Guard (coverage, attribution, and the single-family agreement floor alone clear 50 with evidence three budgets stale), so freshness gates directly and confidence stays a disclosed number. When zero indicators resolve, confidence is 0, not the neutral 0.5 agreement term.
- **KTD-15. Intervals are weight-perturbation sensitivity, labeled as such.** 100 draws per tick (50 when the tick budget is exceeded, disclosed as `intervalDraws`): pillar weights ±20% renormalized, indicator weights ±20%, goalposts ±10%; p05/p95 of Level; `intervalMethodology: 'weight-perturbation-sensitivity-v1'`; `rankStable` when width ≤ 10. Input-data uncertainty is not claimed, and every surface that shows the range labels it "sensitivity range (weights and goalposts), not a confidence interval".
- **KTD-16. History is a per-country Redis ZSET with formula-tagged members plus committed monthly snapshots.** `instability:history:v1:`, member `YYYY-MM-DD:`, score encoding date plus Level as CRI does, trimmed to 730, written once per UTC day by the scores seeder with `withRetry`. Each member also stores every indicator's pre-transform input (`{ value, popMillions, state, certainty, observedAt }`), not its normalized value, so a weight, transform, or goalpost change is a pure recompute over stored inputs rather than a re-ingest, and momentum does not re-arm its warm-up on every bump. Reads never splice across versions (AE6) and use bounded score ranges, since 730 members is 24× the CRI precedent that justified unbounded reads. Backfill runs offline over 180 days of GDELT export files (English plus translation, streamed, resumable) and UCDP GED history with point-in-time discipline; the repo has no GCP presence, so BigQuery is an optional accelerator, never a dependency. Monthly frozen snapshots follow `freeze-resilience-ranking.mjs` into `docs/snapshots/`.
- **KTD-17. v9 ships deferred, then shadow, then live, and is never reachable ungated.** Proto and handlers land registered in `DEFERRED_GENERATED_SERVICES` with a temporary category-prefixed `mcp-api-parity` exclusion, and the entitlement entries for the gated routes land in the same unit (U9) so no gateway can ever expose them unregistered. The seeder needs the gateway over HTTPS, so U16 creates `api/instability/v1/[rpc].ts` and removes the deferred entry in the same PR; the seed-secret refresh path computes and persists regardless of `isInstabilityV1Enabled()`, while every read returns `degraded` until the flag flips at Gated → Live. Read handlers serve live → stale → `degraded` and never compute. "Shadow holds" is an operational gate, not a validity gate (validity is U20's): 14 consecutive days that include at least one UCDP candidate release day and at least one CRI ranking republish, each proven processed rather than merely spanned — the shadow report records the UCDP `versions` transition and the CRI `dataVersion` change as seen by the scorer's readers, and asserts that at least one UCDP-sourced indicator value and at least one vulnerability value changed for at least one country as a result, so a no-op scorer cannot pass (annual transitions such as the UNHCR year rollover are covered by a fixture test in U11, not the shadow), with every scheduled tick published, rank-eligible count within ±10% day over day, adjacent-day Jaccard overlap of the rank-eligible set ≥ 0.9 and first-day-to-last-day overlap ≥ 0.8 (a stable count with rotating identities fails), day-over-day Spearman of v9's own ranking ≥ 0.9 (no thrash), no universe-wide source-failure day, and no day on which more than five countries carry the KTD-2 anomaly flag. No correlation against any scored input is part of the gate (UCDP deaths feed Level, so agreement with them proves nothing), and agreement with v8.1 is not a gate either — it shares v9's GDELT inputs and is the scorer being replaced — so both comparisons are explanatory artifacts only.
- **KTD-18. Agents get new tool identities, not mutated fields.** `get_country_instability`, `get_instability_ranking`, `get_instability_history`, `get_instability_methodology`. `get_country_risk` gains an additive `instability` block at cutover and keeps its `cii` block on v8.1 semantics, marked deprecated in the proto comment and tool description, for the same dated 90-day window the v8 endpoint stays reachable (R21); both are removed together after the `Sunset` date. The `instability` block is the compact public projection — the ranking-row fields plus `pillars[]`, `vulnerability` with its status, `confidence`, `methodologyVersion`, `generationId`, and `comparability` — never `drivers`, `sourceStatus`, `overlays`, or history, so a composite tool that is public or free-account cannot expose the tier-1 full record. Swapping the `cii` block in place was rejected: it would silently change construct and universe under unchanged field names on the highest-traffic country tool, and replacing the proto field's type would fail `make breaking`.
- **KTD-20. `v9.0` launches on three validated pillars; state disruption joins at `v9.1`.** Point-in-time validation needs to know when WorldMonitor could have known each record, and no archive of that exists for outages or ruptures before this plan (Radar annotations carry no publication timestamp; the rupture file's only real first-seen time is `reviewedAt`), so the state-disruption pillar cannot be validated on the calibration window without inventing availability dates. It is therefore computed and published from launch with `status: 'unvalidated'`, excluded from Level, weighted coverage, and `rankEligible`, and shown in the deep dive as "not yet in Level"; the Level weights renormalize over the three validated pillars (0.4375 / 0.3125 / 0.25). From U3 onward the seeders record `firstSeenAt` for every outage, rupture, and IDP row, and the pillar enters Level at `v9.1` after 90 days of that history, a U24-style calibration on it, and a pass on the U20 gates for the four-pillar Level — a methodology bump handled by U17's recompute, never a silent change. Rejected: keeping the pillar in Level with proxied `knownAt` dates (validates a model the users do not receive) and demoting only some of its indicators (leaves a pillar with no core).
- **KTD-19. Naming and non-comparability.** The public product keeps the CII name for v9. v8 keeps its name with a methodology caveat and its own version namespace; no interim rename. Because two things named CII answer with different constructs and universes during the overlap, every v9 response, the methodology pages, the changelog, and the frozen snapshots carry an explicit statement that `v8.x` and `v9.x` scores are not comparable and must not be plotted on one axis; `methodologyVersion` is the discriminator.

### High-Level Technical Design

Data flow across both tracks. Solid arrows are new; dashed arrows exist today.

```mermaid
flowchart TB
subgraph sources [Sources]
GX[GDELT 15-min export + translation]
UC[UCDP GED annual + candidate CSV]
OUT[Cloudflare Radar outages]
UN[UNHCR displacement]
IDU[IDMC IDU flows]
SR[state-ruptures.json]
CRI[CRI structural-readiness pillar]
end
subgraph seeders [Railway seeders]
GB[seed-gdelt-bulk-materializer]
US[seed-ucdp-events]
IS[seed-instability-scores]
ID[seed-conflict-intel HAPI IDP section]
end
subgraph keys [Seed-owned keys]
GD[(gdelt:country-daily:v1)]
UM[(conflict:ucdp-country-monthly:v1)]
UE[(conflict:ucdp-events:v1)]
OK[(infra:outages:v1)]
HK[(instability:history:v1:ISO2)]
SK[(instability:ranking:v1 + score:v1:ISO2 live/stale)]
end
subgraph v8 [Track A: intelligence/v1 get-risk-scores]
S8[computeCIIScores v8.1]
R8[(risk:scores:sebuf:v8.1)]
end
subgraph v9 [Track B: instability/v1]
REG[indicator registry]
SC[pillar scorers + Level]
CF[confidence + rankEligible]
MO[momentum]
IV[intervals]
end
GX --> GB --> GD
UC --> US --> UM
US -.-> UE
OUT -.-> OK
IDU --> ID
GD --> S8
UE --> S8
OK --> S8
S8 --> R8
GD --> REG
UM --> REG
OK --> REG
UN --> REG
ID --> REG
SR --> REG
REG --> SC --> CF
HK --> MO
CRI --> CF
SC --> IV
IS --> SK
IS --> HK
CF --> SK
MO --> SK
```

Rollout states for v9 and the gate between each.

```mermaid
stateDiagram-v2
[*] --> Deferred: U9 merged (proto + handlers + tier entries, no gateway)
Deferred --> Shadow: U16 gateway + seeder live (reads degraded, refresh path computing), history accumulating
Shadow --> Gated: 14-day shadow holds per KTD-17, U20 gates green, Tier-1 eligibility bar met
Gated --> Live: read flag + MCP tools + cutover flag on, get_country_risk gains instability block, v8 kept warm
Live --> Live: monthly frozen snapshot + prospective report
Live --> Retired: day 90 with U26 pass, v8 retired (deprecated tombstones kept)
Live --> Live: cohort unmet at day 90 extends dual run
Live --> Gated: prospective threshold breach pauses retirement, rollback review
Gated --> Shadow: any gate regresses
```

### v9 Indicator Registry (initial edition, methodology `v9.0`)

Weights are within-pillar and sum to 1 per pillar. `abs+rate` means the score is `100 × (0.5 × absNorm + 0.5 × rateNorm)` where `absNorm = clip(log1p(x) / log1p(goalAbs))` and `rateNorm = clip(log1p(x / popMillions) / log1p(goalRate))`; goalposts are frozen at calibration from the p97.5 of the 2024–2026 backfill and recorded in the registry. Recency weight for daily-sourced indicators is a continuous decay `w = 0.5^(ageDays / 10)` over a 30-day window (a 1.0/0.4 step at day 8 was rejected because it puts a cliff into every derived series); UCDP-sourced indicators use a flat 90-day window from the monthly aggregate key (KTD-5). `Corr` is the per-event weight `min(NumSources, 5) / 5` averaged into the intensity sum.

| Indicator | Pillar (weight) | Source / class rule | Transform | Attribution |
|---|---|---|---|---|
| `ac_materialConflictIntensity` | armedConflict 0.35 (renormalized to 0.4375 while state disruption is outside Level; w 0.30) | GDELT roots 18/19/20, QuadClass 4; `unmonitored` below monitoring floor | abs+rate of Corr-weighted event sum | iso-code |
| `ac_fatalities` | armedConflict (w 0.30) | UCDP `best` deaths, all types; stable-absence → observed 0 | abs+rate | iso-code |
| `ac_civilianTargeting` | armedConflict (w 0.15) | UCDP one-sided deaths + `deaths_civilians` share of total deaths (UCDP only; GDELT mass-violence events are weighted 2.0 inside `ac_materialConflictIntensity` instead) | linear 0–1 → 0–100 | iso-code |
| `ac_geographicDiffusion` | armedConflict (w 0.10) | distinct GDELT ADM1 codes with material-conflict events (GDELT only; UCDP adm_1 spread is not merged, so every indicator has one source family) | log1p / log1p(goal) | iso-code |
| `ac_persistence` | armedConflict (w 0.10) | share of days in 30 with ≥ 1 material-conflict event | linear | iso-code |
| `ac_actorFragmentation` | armedConflict (w 0.05) | distinct UCDP dyads in window | log1p / log1p(goal) | iso-code |
| `pd_protestIntensity` | politicalDisorder 0.25 (renormalized to 0.3125; w 0.40) | GDELT root 14, QuadClass 3 | abs+rate | iso-code |
| `pd_riotShare` | politicalDisorder (w 0.20) | 145 events / all 14 events | linear | iso-code |
| `pd_stateCoercion` | politicalDisorder (w 0.25) | GDELT 1723, 1724, 173, 175 where `Actor1Type1Code` is `GOV`, `MIL`, or `COP` (the daily aggregate keeps a separate `coercionGov` class for this) | abs+rate | iso-code |
| `pd_protestDiffusion` | politicalDisorder (w 0.15) | distinct ADM1 with protest events | log1p / log1p(goal) | iso-code |
| `sd_governmentDirectedOutages` | stateDisruption — unvalidated in `v9.0`, outside Level (target weight 0.20 at `v9.1`; w 0.40) | `infra:outages:v1` cause `GOVERNMENT_DIRECTED`; severity-weighted days; no first-seen archive exists before U3, so the seeder records `firstSeenAt` per outage from U3 onward | log1p | name → iso-code |
| `sd_nationwideOutages` | stateDisruption (w 0.25) | `infra:outages:v1` severity `TOTAL` days; `firstSeenAt` recorded from U3 onward | log1p | name → iso-code |
| `sd_unconstitutionalRupture` | stateDisruption (w 0.35) | `state-ruptures.json` (coup, self-coup, dissolution, emergency rule) with decay 90 days; `reviewedAt` is the only real first-seen time, so no backfilled row enters a historical score | discrete 0/60/100 | editorial |
| `sd_aviationClosures` | stateDisruption enrichment | `aviation:delays-bootstrap:v2` closures; ~55 airports only | log1p | name → iso-code |
| `hi_newDisplacement` | humanImpact 0.20 (renormalized to 0.25; w 0.45) | IDMC-sourced conflict-driven new displacement via HDX HAPI, 30 days; enters `v9.0` only if the U12 spike proves 30-day granularity (Q3, Q6), else dropped and the pillar reweights | abs+rate | iso-code |
| `hi_civilianFatalities` | humanImpact (w 0.35) | UCDP one-sided + `deaths_civilians` | abs+rate | iso-code |
| `hi_displacementStockDelta` | humanImpact (w 0.20) | UNHCR `totalDisplaced` year-over-year change | log1p of positive delta | iso3 → iso2 |

Every entry also carries `tier`, `license`, `cadence`, `comprehensive`, `coverage`, `attributionPrecision`, and `windowDays` per `IndicatorSpec`, and exactly one `sourceFamily` — composites were split so the registry type stays single-family and the agreement term can pair indicators by construct. The GDELT monitoring floor and the winsor rule are registry constants, not per-indicator rules.

### Scoring sketch (directional guidance, not implementation)

```text
for country in rankableUniverse:
slots = registry.map(ind => resolve(ind, country)) # {state, value|null, certainty, observedAt, attribution}
for pillar in pillars:
obs = slots where state in (observed, stable-absence, incomplete)
pillar.score = Σ w_i·s_i·c_i / Σ w_i·c_i over obs # c_i = certainty (1.0 observed, 0.7 stable-absence, 0.5 incomplete)
pillar.coverage = Σ w_i·c_i over obs / Σ w_i over all core
pillar.status = insufficient if coverage < 0.5
usable = validated pillars with status != insufficient # v9.0: armedConflict, politicalDisorder, humanImpact
stateDisruption is scored and published with status 'unvalidated' and never enters level or rankEligible in v9.0
level, levelStatus = usable.empty ? (null, insufficient-coverage)
: (Σ W_p·pillar.score over usable / Σ W_p over usable, ok)
weightedCoverage = Σ W_p·pillar.coverage
confidence = 100·(0.35·fresh + 0.30·weightedCoverage + 0.20·agree + 0.15·attr)
rankEligible = weightedCoverage ≥ 0.70 and all validated pillars.coverage ≥ 0.5 and fresh ≥ 0.5
and no core armed-conflict slot in source-failure
activity7 = Σ daily-sourced normalized indicators over trailing 7 days # momentum input, KTD-13
momentum = momentumFrom(history[country].activity7, activity7)
vulnerability, vulnerabilityStatus = criPillar(country, 'structural-readiness')
? (coverage ≥ 0.5 ? (100 − score, ok) : (null, low-coverage)) : (null, unavailable)
drivers = trace(slots) # contribution_i = W_p·w_i·s_i·c_i / Σ… in Level points
assert Σ drivers.contribution ≈ level (±0.5) # reconciliation gate
```

### Sequencing

1. **Track A (weeks 1–3):** U1 → U2 → U3 ∥ U4 → U5 → U6 → U7 → U8. U5 cannot merge before U3 and U4 publish in production and U2's baseline artifact is committed.
2. **Track B foundation (weeks 2–6, overlaps A):** U9 → U10 (provisional goalposts) → U24 (calibration corpus, needs U3, U4, and the U12 spike; final goalposts and the floor threshold back into U10's registry) → U11 → **investment gate** → U13 → U14 → U19 ∥ U15 → U16. U12 in parallel after U9, spike first. No goalpost is used in a published score before U24 has produced it; provisional ones are labeled `provisional` in the registry and the methodology key. The investment gate is a read-only dry run of U11's readers over production keys after U3's backfill: it proceeds only if at least 80 countries would clear the coverage and freshness clauses of `rankEligible` and the U5 calibration criteria were met; otherwise Track B pauses at the registry and readers while the data gaps are closed, and v8.1 stays the product. This is the point at which the 15-unit build is decided on evidence rather than on the plan.
3. **History and momentum (weeks 6–10):** U17 (backfill first, then daily rollup) → U18.
4. **Validation and publication (weeks 8–12):** U20 → U21 → U22.
5. **Cutover (week 12+):** U23 after a 14-day Shadow Measurement.
6. **Prospective validation and retirement:** U26 phase A (power analysis, sealer, evaluator) completes before U23; phase B seals episodes from cutover day and evaluates monthly; U25 at day 90 at the earliest, when U26 reports `pass`; extended or paused per F4.

Weeks are ordering guidance, not commitments.

### Sources & Research

- Scorer: `server/worldmonitor/intelligence/v1/get-risk-scores.ts` (`fetchACLEDEvents` ~605, ACLED bucketing ~846, unrest ~1064, conflict ~1076, blend ~1170, floors ~1184, cache keys ~1397, trend TTL ~1405); weights `shared/cii-weights.ts`; version `_risk-config.ts`; universe `_shared.ts#TIER1_COUNTRIES`.
- Dead paths verified: `scripts/seed-internet-outages.mjs` writes `country` (name) while the scorer reads `countryCode`; `chat-analyst-context.ts#buildRiskScores` and `brief-story-context.ts` read `scores|countries` but the payload is `{ciiScores}`; six hardcoded `risk:scores:sebuf:v8` literals (`api/_cii-risk-cache-keys.js`, `.d.ts`, `scripts/_cii-risk-cache-keys.mjs`, `server/_shared/cache-keys.ts`, `shared/bootstrap-tier-keys.js`, `api/_bootstrap-tier-keys.js`); `docs/zh/methodology/cii-risk-scores.mdx` frontmatter still says v7.
- GDELT: `scripts/_conflict-gdelt-bulk.mjs` (61-column parser, QuadClass gate, 21-entry FIPS map), `scripts/seed-gdelt-bulk-materializer.mjs`, `scripts/_gdelt-fetch.mjs`; external contract in Appendix A.
- UCDP: `scripts/seed-ucdp-events.mjs`, `scripts/shared/ucdp-candidate.cjs`, `tests/ucdp-retention-window.test.mjs`; external contract in Appendix A.
- CRI patterns: `server/worldmonitor/resilience/v1/{_indicator-registry,_dimension-scorers,_pillar-membership,_rankable-universe,_dimension-freshness,_source-failure,_indicator-trace,_shared}.ts` (`_language-coverage.ts` was examined and rejected as a certainty input, see U11); `scripts/_resilience-intervals.mjs`, `validate-resilience-sensitivity.mjs`, `benchmark-resilience-external.mjs`, `backtest-resilience-outcomes.mjs`, `freeze-resilience-ranking.mjs`; `tests/resilience-release-gate.test.mts`, `resilience-doc-parity.test.mts`, `resilience-methodology-lint.test.mts`.
- Service scaffolding exemplar: `cf452179c feat(scorecard): add five-factor resilience scorecards (#7398)`; deferred lever `scripts/lib/sebuf-deferred-services.mjs`; gateway `server/gateway.ts#RPC_CACHE_TIER`; `docs/adding-endpoints.mdx`.
- Registry gates: `scripts/railway-services.json`, `tests/railway-services-registry-coverage.test.mts`, `tests/railway-watch-path-audit.test.mjs`, `tests/nixpacks-seeder-import-graph.test.mjs`, `tests/scripts-shared-mirror.test.mjs`, `tests/mcp-api-parity.test.mjs`, `tests/mcp-bootstrap-parity.test.mjs`, `tests/mcp-tools-reference-docs.test.mjs`, `tests/cii-docs-drift.test.mts`, `tests/cii-scoring.test.mts` (protocol hash ~1611, cache-key consumers ~2448).
- Institutional learnings: `docs/solutions/best-practices/weighted-blend-renormalization-turns-corruption-into-a-higher-score.md`, `logic-errors/flag-dark-dimension-still-moves-published-scores.md`, `design-patterns/primary-fallback-inversion-budget-transfer.md`, `design-patterns/percentile-regime-ladder-needs-a-magnitude-floor-not-a-sign-gate.md`, `conventions/verification-grep-must-cover-every-file-type-it-claims.md`, `design-patterns/closed-world-classification-gate-for-config-completeness.md`, `design-patterns/multi-source-freshness-clock-must-reduce-with-min.md`, `design-patterns/mcp-freshness-check-for-a-new-key-stales-the-whole-tool.md`, `logic-errors/health-must-not-grade-an-unconfigured-optional-source.md`, `logic-errors/an-admission-guard-that-ignores-prior-runtime-cost-repeats-the-bug-it-guards.md`, `database-issues/seeder-auxiliary-redis-writes-timeout.md`, `integration-issues/upstash-max-request-size-counts-one-command-and-answers-http-200.md`, `design-patterns/contract-gate-field-names-miss-value-axis.md`.
- Vocabulary: `CONCEPTS.md` — Seed-Owned Key, Content-Age Contract, Source Tag, Read Outcome, Shadow Measurement, Component Slot, Coverage, Imputation Class, Insufficient History, Closed-World Gate, Mutation Proof, Vacuous Guard.
- Issues: #4061 (CII confidence), #4276 (country history epic), #5852 (source-tag window reset), #3725 (disclosure fields), #3738/#3739 (prior v8 fixes).

---

Contributor guide

Open the contributing guide

Research direction

Start with docs/plans/2026-09-02-001-feat-cii-v9-instability-index-plan.md and the repository conventions in AGENTS.md and CONTRIBUTING.md. Read the existing seed-conflict-intel, seed-gdelt-bulk-materializer, CRI engine, getRiskScores, and refresh RPC entry points before selecting an implementation unit. Done means the specified v8 repair and v9 shadow service pass the release gates, expose the required contracts, and cut over consumers only after validation.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend-api-design, data, observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.