koala73 / koala73/worldmonitor

feat(conflict): add DeepStateMap Ukraine territorial-control polygons (licensing gate first)

Open
#6,230 0 comments 0 reactions 0 assignees View on GitHub
area: map feature P2
Dominant language
TypeScript
Stars
86.6k
Forks
13.1k
Avg merge
8h 4m
Merged PRs (30d)
825

Description

## Summary

We track conflict **events** (ACLED, UCDP, GDELT) but not **territorial control**. DeepStateMap
publishes the Ukraine frontline as GeoJSON polygons — occupied, liberated, and contested areas
with timestamps — which is a different and complementary signal: it answers *"did the line move"*,
not *"was there an incident"*.

## Access — measured, and the reason it needs care

Probed 2026-08-05:

| Endpoint | Result |
|---|---|
| `deepstatemap.live/api/history/last` | **`HTTP 200`** · 627 KB · 0.89 s |
| `deepstatemap.live/api/history` | **`HTTP 401`** |

So the latest snapshot is reachable, but the **history API is locked**. That matches
`../shadowbroker`'s own note (`backend/services/geopolitics.py:73`) that the public API is
locked — which is why they do **not** hit the site directly. They read a third-party GitHub
mirror instead:

- `api.github.com/repos/{repo}/git/trees/{ref}` → `raw.githubusercontent.com`
(`geopolitics.py:79,85`), against `cyterat/deepstate-map-data`
- with an optional `DEEPSTATE_MIRROR_COMMIT` pin to make the read immutable

**That mirror is a supply-chain dependency on an individual's repository.** The commit pin is
their mitigation and we should adopt it if we go that route — but the decision of
direct-vs-mirror needs making deliberately, not by default.

## Licensing — resolve before building

DeepStateMap is a volunteer project with its own terms; `../shadowbroker`'s `DATA-ATTRIBUTION.md`
lists it as *"Per-site terms"* rather than a named open license. **Confirm redistribution rights
before shipping this to users**, especially for a commercial product. If the terms do not permit
it, this issue should close as "declined for licensing" rather than ship — that outcome is an
acceptable result here.

## Why it is worth the trouble

- Territorial control is the single most-requested Ukraine signal and the one users actually
look at; event counts are a poor proxy for it.
- It composes with what we already have: frontline movement × our conflict events × military
flight surges is a genuine convergence signal, and we already run a signal-convergence surface.
- It is one fetch of a few hundred KB — cheap once the access and licensing questions are settled.

## Proposed work

1. **Resolve licensing first.** Contact the project if the terms are unclear. Do not build ahead of this.
2. Decide direct (`/api/history/last`) vs. pinned GitHub mirror, and record the reason. Direct is
simpler and avoids the third-party dependency, but gives no history.
3. Seeder writing GeoJSON to Redis with `seed-meta:` freshness, consistent with the fleet.
4. Map layer for control polygons, with a change-since-last-snapshot diff so *movement* is the
surfaced signal rather than a static shape.
5. Attribution per the resolved terms.

## Acceptance criteria

- [ ] Licensing position documented and cleared before any code lands.
- [ ] Control polygons render with a last-updated timestamp.
- [ ] Frontline delta since the previous snapshot is computed and exposed.
- [ ] Source path (direct vs. pinned mirror) recorded with rationale; if mirrored, the commit is pinned.
- [ ] Attribution present (see the attribution page issue).

## Related

- New-sources issue (GFW / EUvsDisinfo / IODA / SWPC) — same source audit.
- Attribution page issue.

Contributor guide

Open the contributing guide

Research direction

Start by resolving and documenting DeepStateMap's redistribution terms; if they do not permit use, close this as declined for licensing. If cleared, review the existing signal-convergence surface and Redis seeding conventions, then compare the direct endpoint with the pinned GitHub mirror; done means the licensing decision, source rationale, timestamped polygons, frontline delta, and attribution are documented and implemented or the issue is closed.

Written by the indexing model from the issue text.

Assessment

Tech stack
redis, typescript
Domain
data-visualization, databases, full-stack
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.