koala73 / koala73/worldmonitor
feat(sources): add Global Fishing Watch, EUvsDisinfo, IODA + RIPE Atlas, and NOAA SWPC
- Dominant language
- TypeScript
- Stars
- 86.6k
- Forks
- 13.1k
- Avg merge
- 8h 4m
- Merged PRs (30d)
- 825
Description
## Summary
A mechanical host-level diff of `../shadowbroker` against this repo (148 upstream hosts;
57 shared, 91 absent here) surfaced four upstreams that add genuinely new signal to a
geopolitical-intelligence product rather than duplicating what we already ingest.
Ordered by value-to-effort. Each was probed live on 2026-08-05 — results below are measured,
not assumed.
---
### 1. Global Fishing Watch — dark-vessel / AIS-gap detection
- `gateway.api.globalfishingwatch.org/v3/events` · free Bearer token (`401` unauthenticated, confirmed)
- Licensed CC BY 4.0 for public data.
**Why:** we already ingest live AIS via AISstream, but we do nothing with *AIS gaps*. A vessel
switching off its transponder is the core shadow-fleet and sanctions-evasion signal, and it is
the single biggest hole in our maritime coverage given the data we already hold. GFW publishes
the encounter/loitering/gap event feed that makes it computable.
Wired in `../shadowbroker` at `backend/services/fetchers/geo.py:305,325,342`.
---
### 2. EUvsDisinfo (EEAS) — FIMI / state disinformation narratives
- `euvsdisinfo.eu/disinformation-cases/feed/` · `200` · keyless RSS
- Refresh 12 h is ample (upstream updates weekly).
**Why:** nothing in our stack covers information operations. This gives debunked narratives,
named threat actors, and target countries from the EU's own External Action Service — a
category of signal adjacent to everything we already do on conflict and influence, and
currently absent.
Wired in `../shadowbroker` at `backend/services/fetchers/fimi.py:20,280`.
---
### 3. IODA (Georgia Tech) + RIPE Atlas — national internet shutdown detection
- `api.ioda.inetintel.cc.gatech.edu/v2/outages/alerts` · keyless (academic terms; the
parameter shape needs checking — a naive `from`/`until` probe returned `400`)
- `atlas.ripe.net/api/v2/anchors/` · `200` · keyless
**Why:** we have Cloudflare Radar, which is one vantage point owned by one company. IODA and
RIPE Atlas are the academic standard for detecting national internet shutdowns and they
corroborate — or contradict — Radar from independent measurement infrastructure. Shutdown
detection is a high-confidence political-event signal and single-vantage detection is fragile.
Wired in `../shadowbroker` at `backend/services/fetchers/infrastructure.py:58,134,143`.
---
### 4. NOAA SWPC — space weather (Kp index, solar events)
- `services.swpc.noaa.gov/json/planetary_k_index_1m.json` · `200` · 28 KB · keyless
- Public domain (US Federal).
**Why:** this is the natural-cause control for our GPSJam layer. Geomagnetic storms degrade
GPS and HF propagation exactly like deliberate jamming does. Without SWPC we have no way to
avoid reporting a solar event as interference — that is a correctness problem in an existing
panel, not just a new feature.
Note we already ingest other NOAA endpoints (NWS, NHC, NCEI, GML) but **not** SWPC.
Wired in `../shadowbroker` at `backend/services/fetchers/earth_observation.py:206,226`.
---
## Suggested sequencing
SWPC first — it is keyless, tiny, and fixes a correctness gap in a panel we already ship.
Then EUvsDisinfo (keyless RSS, low effort, new category). Then GFW (needs a token and real
modelling work, highest ceiling). IODA/RIPE Atlas last — the endpoint contract needs
establishing and the value is corroborative rather than net-new.
## Acceptance criteria
- [ ] Each source has a Railway seeder writing to Redis with `seed-meta:` freshness, consistent with the existing fleet.
- [ ] SWPC Kp is joined to the GPS-interference surface so solar-driven degradation is distinguishable from jamming.
- [ ] GFW AIS-gap events are exposed on the maritime surface.
- [ ] EUvsDisinfo narratives are queryable by target country.
- [ ] Attribution added for each (CC BY for GFW; see the attribution page issue).
## Related
- Same source audit as the ADS-B redundancy and OpenSky quota issues.
- Deliberately **out of scope** here: cameras and road-traffic feeds.
Contributor guide
Research direction
Begin with backend/services/fetchers/earth_observation.py, fimi.py, infrastructure.py, and geo.py, then inspect how Railway seeders write Redis seed-meta freshness. Validate the SWPC-to-GPS interference join first, followed by the other three source integrations; done means all acceptance criteria are met, including attribution and surface or query availability.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- redis, typescript
- Domain
- backend, data, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100