internetarchive / internetarchive/openlibrary-client
Bot Opportunies
- Dominant language
- Python
- Stars
- 506
- Forks
- 110
- PR merge metrics
- No merged PRs in 30d
Description
### Preface
The old [Writing Bots Wiki](https://github.com/internetarchive/openlibrary/wiki/Writing-Bots) used to have a list of "Bot Opportunities" that have been migrated here. The list migrated from the wiki will likely need further breakdown.
Contributors can edit this issue to track all bot-related clean up
### Legend
- A: Authors
- E: Editions
- W: Works
### Bot Clean up Opportunities
- [ ] Unicode [A, E, W]
- 'mojibake' A~(c) style encoding errors. From multiple sources
- Some from Amazon bulk imports
- Some from external MARC records Some are lossy, others can be reconstructed. Important point to note: there are multiple styles that look similar, but require different translation to fix.
- Combining diacritics. Some external MARC records have had subsequent correction, reimportation could catch many such cases internetarchive/openlibrary#150
- NFC Normalisation. Of titles, description, place, i.e. all text fields. Ensure all input mechanisms normalise correctly. UI + import paths. TEST.
- [X] Spaces in OCAIDS [E] internetarchive/openlibrary#467
- [ ] URLs in OCAIDs [E]
- not many?
- [ ] URLs in Amazon ids [E]
- Needs checking. Some url ids appear to resolve correctly. Suspicion that some users are trying to insert affiliate links in OL ids. What is our policy on this? Should we be making more effort to ensure only Internet Archive affiliate links are used?
- [ ] Invalid ISBNs [E]
- _10 and _13 Other ISBN fields; can they be used for anything?
- [ ] repair if possible
- [ ] normalise https://github.com/internetarchive/openlibrary-bots/pull/43#
- [ ] remove if bad
- [ ] test for reuse on multiple editions
- [ ] Invalid/Incorrect other ids [E, W]
- [ ] LCCN internetarchive/openlibrary#2851
- [ ] OCLC
- [ ] LibraryThing internetarchive/openlibrary#497
- Sometimes ids are valid but link to a different book. How to resolve?
- [ ] Empty Titles [A, E, W]
- Repair if possible
- Delete if junk data
- [ ] Titles varied only by punctuation or whitespace (where author is the same) [E, W]
- Remove trailing ".", ":", ";"
- Reduce whitespaces to single sp
- When one version has a single sp and another has no sp, change latter to single sp (e.g. Hitchhiker's guide --> Hitch hiker's guide)
- When one version has embedded punctuation and another does not, change latter to match former (e.g. Hitch hikers guide --> Hitch-hiker's guide)
- [ ] Name Order [A]
- The Open Library convention is natural name order as opposed to "Last, First" Is everyone happy with this standard? How should titles be handled? e.g. Sir , Lord, Lady, Mrs. There seems to be variation in the data for this style of name.
- Also, how to handle Aliases? I notice the author_role type https://openlibrary.org/type/author_role has an 'as' property. Is this to allow for name variations on a particular book?
- [ ] Bad Alternate names for authors [A] internetarchive/openlibrary#498
- [ ] Orphaned Editions [E] internetarchive/openlibrary#153
- [ ] Works without Editions [W] internetarchive/openlibrary#2727
- https://github.com/internetarchive/openlibrary/issues/44 Appears to be a merging task now. These editionless works look to be left over from moving editions (translations are a common example) to another work without converting the original work to a redirect. If there was some way to go back to the the original edition move and complete the work redirection? Perhaps the WorkBot log at https://github.com/internetarchive/openlibrary/blob/629f2afe8fbff9e8af50e00a57e756dff45a0d73/openlibrary/catalog/works/live.py#L438 might be helpful??
- [ ] Editions unlinked to Works [E]
- Appears to occur mainly with apparently editionless works. How common is it? If rare, it should be fairly simple to address manually in librarian mode given a list of such editions.
- [ ] Authors without Work [A]
- [ ] Orphaned editions without Authors [A]
- Where author is known to IA, from MARC records. Update the records from the original MARC. Many of the missing authors seem to be on books that have editors rather than straight authors. Is this a cause of the problem? How to correctly represent editors on Open Library works?
- [ ] Non-book Items internetarchive/openlibrary#2404
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.