internetarchive / internetarchive/openlibrary-bots

Bot Proposal: HTML Encoding Bot

Open
#442 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
81
Forks
67
PR merge metrics
No merged PRs in 30d

Description

In my time editing Open Library records, I've run into a handful of records (Authors, Editions, and Works) that have HTML encoding as part of their names. Often that causes duplicate records - one with the correct name and one with the HTML-encoded name.

I'd like to write a bot that finds and fixes HTML encodings in record names. [Here's a sample](https://openlibrary.org/authors/OL13003231A/Felix_Ladst%C3%A4tter?m=diff&b=2) of the kind of change the bot would be making.

### Technical details:
Loosely, my plan is:

1. Find records with &...; in the name
2. Use a library like [FTFY](https://ftfy.readthedocs.io/en/latest/fixes.html#ftfy.fixes.unescape_html) to get the correct characters
3. Verify that the unescaped name actually changed, then write that change back to Open Library

### Potential Issues

- This may result in text that uses non-normalized letters (see [this sample](https://openlibrary.org/authors/OL10412204A/%E1%B4%87%C9%B4%E1%B4%84%E1%B4%80%C9%B4%E1%B4%9B%E1%B4%8F_9k_Book?m=diff&b=2)). My plan above doesn't account for that. I'm not sure whether fixing that belongs in this bot or a separate bot.
- While unlikely, it's possible there does exist a record that legitimately has an HTML encoding in its name. That would be hard to determine before-the-fact, but this bot could avoid editing any records it had already edited, allowing users to manually correct the name without getting into an edit war with the bot.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.