internetarchive / internetarchive/openlibrary

Make ImportBot archive non-IA/OL sources in Wayback Machine

Open
#8,758 0 comments 0 reactions 0 assignees View on GitHub
Lead: @mekarpeles Priority: 3 Type: Feature Request Wayback
Dominant language
Python
Stars
6.7k
Forks
2k
Avg merge
2d 19h
Merged PRs (30d)
138

Description

### Describe the problem that you'd like solved

![revision history](https://github.com/internetarchive/openlibrary/assets/177659/1665049d-cbca-481e-acd1-19bbad43a96d)

ImportBot imports from other online sources and denote the source using a "record" link, but this record link can be fragile and may disappear at any point. E.g., the record link for the [initial revision of Twelve Years a Slave](https://openlibrary.org/books/OL29225719M/Twelve_Years_a_Slave?m=history) links to https://www.betterworldbooks.com/product/detail/9781910173121 – which doesn’t exist (anymore/currently). My first instinct was to then go to https://web.archive.org/web/2/https://www.betterworldbooks.com/product/detail/9781910173121 but alas, “The Wayback Machine has not archived that URL.”

### Proposal & Constraints

It would be neat if ImportBot, when doing an import, would send off a request to the Save Page Now API of Wayback Machine to keep a copy around of the record that was used to import from for posterity. This would also be good for pages that have already been archived or that might still be around since it would save the resource at that point in time so the import can be checked against what data was there at the time of import.

### Additional context

Wayback Machine’s Save Page Now API is documented at https://archive.org/details/spn-2-public-api-page-docs-2023-01-22 / https://docs.google.com/document/d/1Nsv52MvSjbLb2PCpHlat0gkzw0EvtSgpKHu4mk0MnrA

### Stakeholders

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.