forrtproject / forrtproject/flora-extractor
study_r / study_o must be study identifiers everywhere, never paper titles
- Dominant language
- Python
- Stars
- 2
- Forks
- 1
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 4
Description
FLoRA's codebook defines `study_o` as the **target study number inside the original paper** ("Study number, can be multiple", e.g. `1, 2`), coded from the replication's abstract. Across this repo and the validation DB the same names are used for **paper titles**. One name, two meanings — and the real study numbers had nowhere to go.
`extracted.csv` now has a real `study_o` (numbers), and `extract/csv_to_db.py` sends titles as `title_r` / `title_o` and `study_o` as the number (PR: `audit/pr10-flora-rule-alignment`). The rest is listed here.
### Blocking — validation repo (`flora-validation`)
`csv_to_db` will fail on every insert until the `unvalidated` table has these columns. Nothing is written and nothing is corrupted in the meantime, but **the import is unusable until this lands**.
- [ ] `ALTER TABLE unvalidated ADD COLUMN title_r text, ADD COLUMN title_o text;`
- [ ] Backfill `title_r` / `title_o` from the existing `study_r` / `study_o` values (they currently hold titles).
- [ ] Clear `study_r` / `study_o` after the backfill, then let `csv_to_db` repopulate `study_o` with study numbers. `study_r` stays empty — see below.
- [ ] Update the validator UI to read the title from `title_r` / `title_o` and to show `study_o` as "Study 2 of the original", not as a title.
- [ ] `db_schema.sql` comments on both columns saying they are identifiers.
### This repo — legacy readers that treat `study_r` as a title
These read seeded CSVs (`all_replications.csv`, `flora_selected.csv`, old-pipeline exports) whose `study_r` column really does hold a title, so they are not wrong today — they are the reason the confusion spread. They need renaming to `title_r` at the same time as the CSVs they read.
- [ ] `validate/routes/batch.py:74,84,312`
- [ ] `validate/routes/multi_originals.py:35,61`
- [ ] `validate/routes/dashboard.py:555,581,585`
- [ ] `validate/templates/batch.html:314,381`
- [ ] `validate/templates/multi_originals.html:149,215`
- [ ] `validate/templates/dashboard.html:3216,3246`
- [ ] `validate/static/render.js:92,95,169,272` — line 95 falls back to `data.study_o` for a title, which is exactly the conflation
- [ ] `analysis/old_pipeline_compare.py:182,215`
- [ ] `extract/run_extract.py:_merge_row` — `title_r` falls back to `row["study_r"]` for old seeded data; drop once the seeds are renamed
### Open question — is there a `study_r` at all?
FLoRA's codebook has no `study_r`; the `study_o` row explicitly notes "Not study_r". The validation DB has the column. Either it means "which study of the replication paper" (which the pipeline does not code, so it would stay empty) or it should be dropped. Worth settling before the backfill above.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with validation/db_schema.sql, validation's unvalidated table, and extract/csv_to_db.py to confirm the intended column meanings. Then trace the listed batch, multi-originals, dashboard, template, render, and legacy pipeline readers, including the study_o title fallback. Done means the schema, backfill, UI labels, seeded CSV readers, and imports consistently keep titles in title_r/title_o and study_o as a number, after deciding the fate of study_r.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- html, javascript, python, sql
- Domain
- backend, databases, frontend
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100