internetarchive / internetarchive/openlibrary

Records being created with undecoded HTML entities in names

Open
#7,350 0 comments 0 reactions 0 assignees View on GitHub
Lead: @mekarpeles Module: Import Priority: 3 Theme: Unicode Type: Bug
Dominant language
Python
Stars
6.7k
Forks
2k
Avg merge
2d 19h
Merged PRs (30d)
138

Description

I've seen a number of examples where the names of authors and books include undecoded HTML entities, such as:
https://openlibrary.org/authors/OL2743682A/Ren%C3%A9_Thi%C3%A9blemont?b=3&a=1&_compare=Comparer&m=diff

### Evidence / Relevant URLs

Other examples can be found by searching for the names of HTML entities, e.g.
https://openlibrary.org/search/authors?q=eacute&mode=everything

https://openlibrary.org/authors/OL2743682A/Ren%C3%A9_Thi%C3%A9blemont?b=3&a=1&_compare=Comparer&m=diff

### Expected Behavior

All XML entities are decoded on input before creating an OpenLibrary record

### Proposal & Constraints

Steps to resolve:
- Figure out root cause of the problem
- Fix the import pipeline if it hasn't already been fixed
- Replace HTML entities with decoded character in all record names
- Merge any obvious duplicates that the newly correct names reveal

### Related files

### Stakeholders
@hornc @mekarpeles

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.