eduaguilera / eduaguilera/whep

country_areas.parquet spatializes four FAO region aggregates alongside their own members

Open
#656 1 comment 0 reactions 0 assignees View on GitHub
area:regions area:spatialize bug needs-expert priority:medium
Dominant language
R
Stars
1
Forks
5
Avg merge
1d 11h
Merged PRs (30d)
186

Description

**Problem**

`country_areas.parquet` — the national table `build_gridded_landuse()`
allocates *from* — carries four FAO **region aggregates** alongside their own
member countries:

| `area_code` | area | harvested area, summed over its span | span |
|---|---|---|---|
| 904 | Latin America Other | 9,756,143 ha | 1851-2006 |
| 906 | Oceania Other | 3,068,117 ha | 1851-2023 |
| 901 | Africa Other | 2,093,147 ha | 1961-2006 |
| 903 | Europe Other | 21,598 ha | 1851-2023 |

Measured on the deployed pin at `WHEP_*` paths (1,032,750 rows, 198 codes),
`origin/main` at `bd225575`.

This is the half of #582 that #582 explicitly deferred: *"the region
aggregates being present at all deserves a look"*, and *"The FAO region
aggregates need a separate call: drop, or keep and document why a region total
is being spatialized alongside its own members."* whep#653 fixed the
disagreeing-key half and deliberately left this one, because the four codes
**are** their own `polity_area_code`, so the invariant that PR added cannot
see them.

**Why it is not obviously harmless**

None of the four has a `country_grid` cell (they are in the 21-code zero-cell
set `.warn_unallocated_crops()` already surfaces), so today they are warned
about and dropped rather than double-counted. Two things make that worth
deciding rather than leaving:

1. If a future `country_grid` vintage ever rasterizes a region aggregate — or
if a consumer aggregates `country_areas` directly instead of the gridded
output — the region total sits beside its own members and double-counts.
2. The drop is currently indistinguishable, in the warning, from a genuine
small-island country that has no cell. One is a data-model error, the other
is #461's centroid-coverage question.

**What needs deciding**

Whether `prepare_spatialize_all.R` should filter FAO region aggregates out of
`country_areas` before writing the pin, or keep them with a documented reason
and a marker column so consumers can tell an aggregate from a country. Note
that `get_primary_production()` does **not** publish these four codes at all
today (verified for 2020: 208 codes, none of `901/903/904/906`), so the pin's
vocabulary is already wider than production's — see the sibling issue about
the pin's vintage.

**Acceptance criteria**

- The decision is stated in `prepare_spatialize_all.R` or the pin's
documentation, not implied.
- If they are kept, a test asserts they cannot be allocated to a cell that
also receives one of their members.

Part of the polity migration epic #458.

Contributor guide

Open the contributing guide

Research direction

Start with prepare_spatialize_all.R and the pin documentation, then inspect how region aggregates reach country_areas and how .warn_unallocated_crops() reports zero-cell entries. Decide whether codes 901, 903, 904, and 906 are filtered or retained with a marker. Done means the decision is documented; if retained, add the required allocation-safety test.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data-engineering
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.