internetarchive / internetarchive/openlibrary
find_work_for_edition misses variant author spellings, causing redundant works
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 2k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 138
Description
This does not check for variant spellings of authors when multiple author records exist for a given author:
https://github.com/internetarchive/openlibrary/blob/7afe9d853bdb282a70c11ff80b2a3bba874e61ac/openlibrary/catalog/works/find_work_for_edition.py#L14
The direct result is that often there is no matching work returned when there should be. The indirect result is that new work records are created for works that already have work records attributed to different author records for the same author!
Similarly, variations in title spelling (accents, capitalization, or omission of stopwords) seem to cause the same effect.
Before creating new work records a much more aggressive check for existing records is needed. Just looking for perfect matches of a record identifier is a huge waste of human effort.
Contributor guide
Assessment
This issue has not been assessed yet.