forrtproject / forrtproject/fred-data

reported_success back-filled with numeric junk from the main sheet's `outcome` column (43 rows)

Open
#132 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
1
Forks
0
PR merge metrics
No merged PRs in 30d

Description

## Problem

`prepare_fred.qmd:570` back-fills `reported_success` from the main FReD Google Sheet's `outcome` column:

```r
reported_success = coalesce(reported_success, outcome),
```

That column does not contain outcome codes. Of its 3,109 non-NA values:

| shape | n |
|---|---|
| purely numeric (`1`×1018, `124`×262, `10`×183, `92`×172, `31`×102, `2`×104, …) | 2,256 |
| `#N/A` (broken spreadsheet formula) | 839 |
| long free text (>60 chars, e.g. "independent predictors of forgiveness") | 4 |
| **actual outcome labels** | **2** (one `failed`, one `successful`) |

So wherever the COS success-coding join leaves `reported_success` empty, the pipeline substitutes an unrelated number.

## Impact

A fresh render of `prepare_fred.qmd` (2026-07-25) produces **43 rows** with numeric `reported_success`: `31`×19, `12`×12, `1`×10, `124`×2.

Affected `fred_id`s include all the `XPHI*` entries (`XPHI1`, `XPHI9`, `XPHI17`, `XPHI18`, `XPHI20`, `XPHI21`, `XPHI24`–`XPHI27`, `XPHI29`), `additional_studies4/40/48/76/78/91`, `Boyce_etal2023137_1`, `Boyce_etal202368`, `CORE_147`, `curatescience23`.

Note their `reported_success_quote_source` is populated correctly (`abstract`, `https://osf.io/4ewkh`, …), so only the code itself is wrong — the provenance columns are fine.

## How it was found

The outcome-vocabulary normalisation added in 01d5e75 includes a check that warns when `reported_success` falls outside the 15 allowed values. It fired on the first fresh render:

```
⚠ reported_success values outside the allowed vocabulary: '1', '12', '124', '31'
```

This is pre-existing — the `coalesce` predates that commit. It did not show in the committed `output/FReD.xlsx` (last built 2026-02-01) because the source sheet has drifted since.

`output/FReD.xlsx` was **not** updated with the bad render; it remains at the 2026-02-01 build.

## Options

1. **Guard the coalesce** — only fall back to `outcome` when the value is in `VALID_OUTCOMES` (see `R/outcome_vocabulary.R`). The 43 rows become `NA`, which is honest; costs the 2 genuine labels unless they are re-coded at source.
2. **Drop the coalesce entirely** — the column yields 2 usable values out of 3,109, so the fallback arguably has no value at all.
3. **Fix the source sheet** — work out what the `outcome` column is actually holding (the numbers look like counts or IDs), rename it so it stops colliding with the outcome vocabulary, and repair the `#N/A` formula.

Option 1 or 2 unblocks re-rendering FReD; option 3 is the durable fix.

## Also worth noting

A fresh render also changes the row count 2,164 → 2,036 and leaves 33 rows with no `reported_success` at all. That is ~6 months of upstream sheet drift rather than a bug, but it means the next `FReD.xlsx` refresh is a substantive data change that should be reviewed on its own.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at prepare_fred.qmd:570 and read R/outcome_vocabulary.R, then render the pipeline to reproduce the invalid reported_success warning. Decide whether to guard or remove the fallback from outcome, and verify that numeric junk is no longer produced while valid outcome labels and provenance columns remain intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
64/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.