internetarchive / internetarchive/openlibrary

Stop creating bad author records from BWB import

Open
#7,756 1 comment 0 reactions 0 assignees View on GitHub
Lead: @jimchamp Module: Import Priority: 3 Type: Bug
Dominant language
Python
Stars
6.7k
Forks
2k
Avg merge
2d 19h
Merged PRs (30d)
138

Description

In the last 12 months, there has been a huge increase in the creation of corrupted author records, almost always from BWB "promise" or "pallet" imports. The bad records come in a variety of forms including containing multiple author names in a single record, containing roles in the name (e.g. editor, translator), all lower case, etc. Also, since BWB authors apparently never include dates, strong identifiers, or any other type of disambiguating information, a large number of duplicate author records are being created.

Below are two different corrupted forms create for the same pair of authors, both of whom already have existing records in OpenLibrary. Not only do the author records exist, but this exact [edition](https://openlibrary.org/books/OL13501885M) was already cataloged and scanned 15 years ago, but it's impossible to match due to metadata corruption.

### Evidence / Screenshot (if possible)
[Thaddeus Eddy Samuel; Surber](https://openlibrary.org/authors/OL12302678A/Thaddeus_Eddy_Samuel_Surber)
![Screen Shot 2023-04-03 at 6 58 15 PM](https://user-images.githubusercontent.com/82178/229649883-ad2b71f1-84a2-4f41-bcfa-5e147c29ab4f.png)

[Eddy, Samuel Surber, Thaddeus,](https://openlibrary.org/authors/OL12264401A/Eddy_Samuel_Surber_Thaddeus)
![Screen Shot 2023-04-03 at 6 58 26 PM](https://user-images.githubusercontent.com/82178/229649897-bcbf8504-3663-4856-8448-7e2172fe8664.png)

### Relevant url?

https://openlibrary.org/books/OL45868829M
https://openlibrary.org/books/OL45991226M

### Proposal & Constraints

The BWB importer should be banned from creating new author records until it can do so with a quality on par with those created from MARC records.

New metadata sources should undergo a quality audit before being integrated into the production system.

### Stakeholders
@mekarpeles

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.