internetarchive / internetarchive/openlibrary
Data Analysis of Potential Duplicate Authors
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 2k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 138
Description
### Proposal
[Here](https://internetarchive.slack.com/archives/C0ETZV72L/p1738991328546269?thread_ts=1738954311.514009&cid=C0ETZV72L) we have a list of 100k+ authors that have the exact same name and IDs that are only one off from each other. They are likely to be duplicate authors because of a race condition. As of 2024 there are few instances of this problem. But we still need to fix up the old instances.
Here's what someone should do:
1. For every single author get the number of works and if they only have 1 work (it seems most do) get the title of that work. Upload that here as a CSV like the one already attached but with these two additional fields.
2. From that, we should produce a list of all pairs of authors that have identical names, works with identical (case in-sensitive) titles, and IDs off by one.
From there staff can decide if we want to do an automated merge.
To work on this please use the data dumps, do not call the API.
CSV of authors in case you can't see the file on slack:
[ids_next_to_each_other.csv](https://github.com/user-attachments/files/18756768/ids_next_to_each_other.csv)
### Justification
_No response_
### Breakdown
#### Requirements Checklist
* [ ]
#### Related files
*
#### Stakeholders
*
#### Instructions for Contributors
Please [run these commands](https://github.com/internetarchive/openlibrary/wiki/Git-Cheat-Sheet#working-on-your-branch) to ensure your repository is up to date **before** [creating a new branch](https://github.com/internetarchive/openlibrary/wiki/Git-Cheat-Sheet#making-changes-and-creating-a-pull-request) to work on this issue and **each time after** pushing code to Github, because the pre-commit bot may add commits to your PRs upstream.
Contributor guide
Assessment
This issue has not been assessed yet.