eduaguilera / eduaguilera/whep

biomass_coefs: 72 rows have proximate mass above their dry matter, and one Edible_portion is 4.0

Open
#752 3 comments 0 reactions 1 assignee Claimed by @lbm364dl View on GitHub
area:data-io bug needs-expert no-data-needed priority:medium
Dominant language
R
Stars
1
Forks
5
Avg merge
1d 11h
Merged PRs (30d)
183

Description

## What is wrong

Two internal-consistency violations in
`inst/extdata/harmonization/biomass_coefs.csv`, measured on current `main`
(421 rows).

### 1. 72 rows carry more proximate mass than dry matter

Summing protein (`N_kgN_kgFM` x 6.25), `Carbohydrates_g_kgFM`,
`Lipids_g_kgFM` and `Fiber_g_kgFM` and comparing against
`Product_kgDM_kgFM` x 1000:

| Name_biomass | proximate sum (g/kg) | dry matter (g/kg) | excess |
|---|--:|--:|--:|
| Urea | 2846.2 | 990.0 | 1856.2 |
| Carob | 1245.5 | 718.0 | 527.5 |
| Tigernuts | 723.0 | 300.3 | 422.7 |
| Figs | 610.0 | 197.0 | 413.0 |
| Lysine | 1234.4 | 980.0 | 254.4 |
| Honey | 1000.0 | 785.0 | 215.0 |
| Cocoa beans | 919.0 | 718.0 | 201.0 |
| Barley | 1068.5 | 885.0 | 183.5 |
| Hemp seed | 1085.6 | 905.5 | 180.1 |

Some rows are feed additives where a composition block is simply not
applicable (Urea, Lysine). But ordinary foods are affected too — Barley, Figs,
Carob, Honey, Cocoa beans — so this is not only a not-applicable-row artifact.

The practical consequence is that the proximate columns cannot be used to
reason about a row's basis. Concretely, in #500 the argument "Fiber_g_kgFM =
18.5 is too low for hull-bearing rough rice, so the Rice row must be milled"
does not hold, because that row's own proximate columns sum to 961.8 g/kg
against 864 g/kg of dry matter. The conclusion happens to be right, but the
evidence is not usable.

### 2. One `Edible_portion` outside (0, 1]

Exactly one row: `ANIMAL PRODUCTS`, `Edible_portion = 4.0`.

**This is not a leaked section header, and it must not be "cleaned" upstream.**
In `afsetools/inst/extdata/Biomass_coefs.xlsx` that row is the VLOOKUP
column-index vector the whole `Coefs` sheet depends on by absolute address
(row 292: M=2, S=4, T=3, U=5, V=6, W=7, X=8). Deleting or editing it upstream
breaks the workbook. It should be filtered downstream at ingestion instead.

It is currently harmless — no `item_cbs_code` in `items_full.csv` maps to
`Name_biomass == "ANIMAL PRODUCTS"`, so `build_food_supply()` cannot reach it —
but nothing asserts that, so a future mapping edit could silently pull a
4x edible fraction into a food calculation.

## Suggested resolution

Add ingestion-time invariants rather than editing the upstream workbook:

- `0 < Edible_portion <= 1`, with `ANIMAL PRODUCTS` filtered at read and
asserted unreachable from any `item_cbs_code`.
- proximate sum `<=` dry matter, scoped to rows where a composition block is
meaningful, so the exemptions are explicit rather than implied.

Deciding which of the 72 rows carry a wrong value (as opposed to a
not-applicable one) is a coefficient question for the data owner, which is why
this is `needs-expert` rather than `mechanical`.

Reproduced on `main` with `inst/extdata/harmonization/biomass_coefs.csv`; no
pin or network needed.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.