koala73 / koala73/worldmonitor

feat(sources): add Global Fishing Watch, EUvsDisinfo, IODA + RIPE Atlas, and NOAA SWPC

Open
#6,225 0 comments 0 reactions 0 assignees View on GitHub
area: feeds area: panels feature P2
Dominant language
TypeScript
Stars
86.6k
Forks
13.1k
Avg merge
8h 4m
Merged PRs (30d)
825

Description

## Summary

A mechanical host-level diff of `../shadowbroker` against this repo (148 upstream hosts;
57 shared, 91 absent here) surfaced four upstreams that add genuinely new signal to a
geopolitical-intelligence product rather than duplicating what we already ingest.

Ordered by value-to-effort. Each was probed live on 2026-08-05 — results below are measured,
not assumed.

---

### 1. Global Fishing Watch — dark-vessel / AIS-gap detection

- `gateway.api.globalfishingwatch.org/v3/events` · free Bearer token (`401` unauthenticated, confirmed)
- Licensed CC BY 4.0 for public data.

**Why:** we already ingest live AIS via AISstream, but we do nothing with *AIS gaps*. A vessel
switching off its transponder is the core shadow-fleet and sanctions-evasion signal, and it is
the single biggest hole in our maritime coverage given the data we already hold. GFW publishes
the encounter/loitering/gap event feed that makes it computable.

Wired in `../shadowbroker` at `backend/services/fetchers/geo.py:305,325,342`.

---

### 2. EUvsDisinfo (EEAS) — FIMI / state disinformation narratives

- `euvsdisinfo.eu/disinformation-cases/feed/` · `200` · keyless RSS
- Refresh 12 h is ample (upstream updates weekly).

**Why:** nothing in our stack covers information operations. This gives debunked narratives,
named threat actors, and target countries from the EU's own External Action Service — a
category of signal adjacent to everything we already do on conflict and influence, and
currently absent.

Wired in `../shadowbroker` at `backend/services/fetchers/fimi.py:20,280`.

---

### 3. IODA (Georgia Tech) + RIPE Atlas — national internet shutdown detection

- `api.ioda.inetintel.cc.gatech.edu/v2/outages/alerts` · keyless (academic terms; the
parameter shape needs checking — a naive `from`/`until` probe returned `400`)
- `atlas.ripe.net/api/v2/anchors/` · `200` · keyless

**Why:** we have Cloudflare Radar, which is one vantage point owned by one company. IODA and
RIPE Atlas are the academic standard for detecting national internet shutdowns and they
corroborate — or contradict — Radar from independent measurement infrastructure. Shutdown
detection is a high-confidence political-event signal and single-vantage detection is fragile.

Wired in `../shadowbroker` at `backend/services/fetchers/infrastructure.py:58,134,143`.

---

### 4. NOAA SWPC — space weather (Kp index, solar events)

- `services.swpc.noaa.gov/json/planetary_k_index_1m.json` · `200` · 28 KB · keyless
- Public domain (US Federal).

**Why:** this is the natural-cause control for our GPSJam layer. Geomagnetic storms degrade
GPS and HF propagation exactly like deliberate jamming does. Without SWPC we have no way to
avoid reporting a solar event as interference — that is a correctness problem in an existing
panel, not just a new feature.

Note we already ingest other NOAA endpoints (NWS, NHC, NCEI, GML) but **not** SWPC.

Wired in `../shadowbroker` at `backend/services/fetchers/earth_observation.py:206,226`.

---

## Suggested sequencing

SWPC first — it is keyless, tiny, and fixes a correctness gap in a panel we already ship.
Then EUvsDisinfo (keyless RSS, low effort, new category). Then GFW (needs a token and real
modelling work, highest ceiling). IODA/RIPE Atlas last — the endpoint contract needs
establishing and the value is corroborative rather than net-new.

## Acceptance criteria

- [ ] Each source has a Railway seeder writing to Redis with `seed-meta:` freshness, consistent with the existing fleet.
- [ ] SWPC Kp is joined to the GPS-interference surface so solar-driven degradation is distinguishable from jamming.
- [ ] GFW AIS-gap events are exposed on the maritime surface.
- [ ] EUvsDisinfo narratives are queryable by target country.
- [ ] Attribution added for each (CC BY for GFW; see the attribution page issue).

## Related

- Same source audit as the ADS-B redundancy and OpenSky quota issues.
- Deliberately **out of scope** here: cameras and road-traffic feeds.

Contributor guide

Open the contributing guide

Research direction

Begin with backend/services/fetchers/earth_observation.py, fimi.py, infrastructure.py, and geo.py, then inspect how Railway seeders write Redis seed-meta freshness. Validate the SWPC-to-GPS interference join first, followed by the other three source integrations; done means all acceptance criteria are met, including attribution and surface or query availability.

Written by the indexing model from the issue text.

Assessment

Tech stack
redis, typescript
Domain
backend, data, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.