internetarchive / internetarchive/openlibrary

find_work_for_edition misses variant author spellings, causing redundant works

Open
#667 7 comments 1 reaction 0 assignees View on GitHub
Lead: @scottbarnes Module: Identifier Resolution Module: Import Priority: 2 Type: Bug
Dominant language
Python
Stars
6.7k
Forks
2k
Avg merge
2d 19h
Merged PRs (30d)
138

Description

This does not check for variant spellings of authors when multiple author records exist for a given author:

https://github.com/internetarchive/openlibrary/blob/7afe9d853bdb282a70c11ff80b2a3bba874e61ac/openlibrary/catalog/works/find_work_for_edition.py#L14

The direct result is that often there is no matching work returned when there should be. The indirect result is that new work records are created for works that already have work records attributed to different author records for the same author!

Similarly, variations in title spelling (accents, capitalization, or omission of stopwords) seem to cause the same effect.

Before creating new work records a much more aggressive check for existing records is needed. Just looking for perfect matches of a record identifier is a huge waste of human effort.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.