eduaguilera / eduaguilera/whep
country_areas.parquet spatializes four FAO region aggregates alongside their own members
- Dominant language
- R
- Stars
- 1
- Forks
- 5
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 186
Description
**Problem**
`country_areas.parquet` — the national table `build_gridded_landuse()`
allocates *from* — carries four FAO **region aggregates** alongside their own
member countries:
| `area_code` | area | harvested area, summed over its span | span |
|---|---|---|---|
| 904 | Latin America Other | 9,756,143 ha | 1851-2006 |
| 906 | Oceania Other | 3,068,117 ha | 1851-2023 |
| 901 | Africa Other | 2,093,147 ha | 1961-2006 |
| 903 | Europe Other | 21,598 ha | 1851-2023 |
Measured on the deployed pin at `WHEP_*` paths (1,032,750 rows, 198 codes),
`origin/main` at `bd225575`.
This is the half of #582 that #582 explicitly deferred: *"the region
aggregates being present at all deserves a look"*, and *"The FAO region
aggregates need a separate call: drop, or keep and document why a region total
is being spatialized alongside its own members."* whep#653 fixed the
disagreeing-key half and deliberately left this one, because the four codes
**are** their own `polity_area_code`, so the invariant that PR added cannot
see them.
**Why it is not obviously harmless**
None of the four has a `country_grid` cell (they are in the 21-code zero-cell
set `.warn_unallocated_crops()` already surfaces), so today they are warned
about and dropped rather than double-counted. Two things make that worth
deciding rather than leaving:
1. If a future `country_grid` vintage ever rasterizes a region aggregate — or
if a consumer aggregates `country_areas` directly instead of the gridded
output — the region total sits beside its own members and double-counts.
2. The drop is currently indistinguishable, in the warning, from a genuine
small-island country that has no cell. One is a data-model error, the other
is #461's centroid-coverage question.
**What needs deciding**
Whether `prepare_spatialize_all.R` should filter FAO region aggregates out of
`country_areas` before writing the pin, or keep them with a documented reason
and a marker column so consumers can tell an aggregate from a country. Note
that `get_primary_production()` does **not** publish these four codes at all
today (verified for 2020: 208 codes, none of `901/903/904/906`), so the pin's
vocabulary is already wider than production's — see the sibling issue about
the pin's vintage.
**Acceptance criteria**
- The decision is stated in `prepare_spatialize_all.R` or the pin's
documentation, not implied.
- If they are kept, a test asserts they cannot be allocated to a cell that
also receives one of their members.
Part of the polity migration epic #458.
Contributor guide
Research direction
Start with prepare_spatialize_all.R and the pin documentation, then inspect how region aggregates reach country_areas and how .warn_unallocated_crops() reports zero-cell entries. Decide whether codes 901, 903, 904, and 906 are filtered or retained with a marker. Done means the decision is documented; if retained, add the required allocation-safety test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100