internetarchive / internetarchive/openlibrary

Clean up records with future publication dates

Open
#4,568 22 comments 0 reactions 0 assignees View on GitHub
Affects: Data Can it be closed? Data Cleanup Lead: @scottbarnes Needs: Feedback Priority: 3 Type: Bug
Dominant language
Python
Stars
6.7k
Forks
2k
Avg merge
2d 19h
Merged PRs (30d)
138

Description

### Evidence / Screenshot (if possible)
Many works have wrong year of publication (Like 9999, 2049, 2040....)

See: https://openlibrary.org/search?q=publish_year%3A%5B2025+TO+*%5D

![image](https://user-images.githubusercontent.com/64412143/107536659-06775280-6be8-11eb-8991-3b1a81c6589c.png)

### Relevant url?

https://openlibrary.org/search?q=mark&mode=everything&sort=new
https://openlibrary.org/works/OL21132031W/Classical_Music_Picture_Book?edition=
https://openlibrary.org/works/OL21486637W/Making_Sense_of_Politics?edition=

### Details

- **Logged in (Y/N)?** Y
- **Browser type/version?** Chrome
- **Operating system?** Windows 10
- **Environment (prod/dev/local)?** prod

### Proposal

Use `first_publish_year:[2025 TO *]` in solr, e.g. https://openlibrary.org/search.json?q=first_publish_year%3A%5B2025+TO+*%5D, to find future dates

* Run a [json solr query](https://openlibrary.org/search.json?q=first_publish_year%3A%5B2025+TO+*%5D) to get all ~12k records with future dates
* Write a small [script](https://github.com/internetarchive/openlibrary-bots/blob/master/BWBImportBot/import-ol.py#L10-L14) to fetch and update these records by removing the future dates

### Stakeholders

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.