internetarchive / internetarchive/openlibrary
Edition-level `authors` silently corrupted (likely by 2014 author merges) — field is invisible and uneditable in the UI
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 2k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 138
Description
### Problem
Edition-level `authors` links can be silently corrupted (in this case, apparently by 2014 author-merge operations) and the UI provides no way to see or fix the field — the book page and edit form both show only the *work's* authors. The bad data is invisible on openlibrary.org but is served to all API and bulk-dump consumers, who inherit it into downstream apps and datasets.
Concrete case — edition [OL18241658M](https://openlibrary.org/books/OL18241658M) (*Shiloh*, ISBN 0689316143). In [the edition JSON](https://openlibrary.org/books/OL18241658M.json):
- `authors` → [`/authors/OL27452A`](https://openlibrary.org/authors/OL27452A.json) = **Jean Little** ❌
- the same record's `by_statement` = `"by Phyllis Reynolds Naylor."`
- its work [OL13489W](https://openlibrary.org/works/OL13489W.json) correctly links [OL22786A](https://openlibrary.org/authors/OL22786A.json) = **Phyllis Reynolds Naylor** ✅
Version history shows the link was correct from the 2008 import, then rewritten twice in 2014 — consistent with collateral damage from author merges (a misfired merge rewrites *every* edition that pointed at the merged record, so this is unlikely to be a single record):
| revision | date | edition `authors` |
|---|---|---|
| [v4](https://openlibrary.org/books/OL18241658M.json?v=4) | 2010-08-18 | OL22786A (Naylor) ✅ |
| [v5](https://openlibrary.org/books/OL18241658M.json?v=5) | 2014-05-17 | OL1122551A (Paul Galdone) ❌ |
| [v6](https://openlibrary.org/books/OL18241658M.json?v=6) | 2014-06-27 | OL27452A (Jean Little) ❌ — unchanged for 11 years since |
Potential scope: the class is machine-detectable from the dump alone (edition `authors` disagreeing with the work's `authors`; here it even contradicts its own `by_statement`). Cross-referencing a 3,000-edition sample against external sources (Wikidata, LoC, HathiTrust, K10plus) with a data-quality tool I've been building, ~1.3% of editions flagged author disagreements; this record is the first hand-verified as edition-link corruption. Happy to run a broader scan and report back.
(Same record: `publish_date` "1996" also contradicts its own LC classification `PZ7.N24 Sg 1991` and external sources, which agree on 1991.)
### Reproducing the bug
1. Open https://openlibrary.org/books/OL18241658M.json → `authors` is `/authors/OL27452A` (Jean Little), while `by_statement` says "by Phyllis Reynolds Naylor."
2. Open https://openlibrary.org/books/OL18241658M/Shiloh → the page shows "Phyllis Reynolds Naylor" (the work's author), so the error is invisible on the site.
3. Open the edition's edit form → there is no field exposing the edition-level `authors` link, so it can't be corrected through the UI.
* Expected behavior: the edition's author link matches the work/`by_statement` (Phyllis Reynolds Naylor), or at minimum is visible and editable so librarians can fix it.
* Actual behavior: the edition serves Jean Little to API/dump consumers; the website hides the field and the edit form can't touch it.
### Context
- Browser (Chrome, Safari, Firefox, etc): n/a — data-layer issue, reproducible via public JSON API
- OS (Windows, Mac, etc): n/a
- Logged in (Y/N): N (visible logged out)
- Environment (prod, dev, local): prod
### Breakdown
Implementation Details (for maintainers)
Possible directions (happy to leave the call to maintainers):
1. Repair this record: edition `authors` → OL22786A (and `publish_date` → 1991).
2. Size the problem: scan the editions dump for edition-vs-work author disagreements.
3. Longer term: either expose edition-level `authors` in the edit UI, or declare work authors canonical and sync/clear stale edition-level links so API/dump consumers aren't served data the site itself ignores.
#### Requirements Checklist
* [ ]
#### Related files
*
#### Stakeholders
*
#### Instructions for Contributors
- Please [run these commands](https://docs.openlibrary.org/developers/tools/git.html#working-on-your-branch) to ensure your repository is up to date **before** [creating a new branch](https://docs.openlibrary.org/developers/tools/git.html#making-changes-and-creating-a-pull-request) to work on this issue and **each time after** pushing code to Github, because the pre-commit bot may add commits to your PRs upstream.
Contributor guide
Research direction
Start with the edition JSON for OL18241658M, its work JSON, the public book page, and the edition edit form to trace where edition-level authors are stored, served, and hidden. Compare this with the editions dump and author-merge behavior before choosing a scope; done requires an agreed repair, scan, or UI/data-model change with tests or validation for the selected path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend, databases
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100