internetarchive / internetarchive/openlibrary
Check other regions when importing from Amazon, not just US
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 2k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 138
Description
### Feature Request
(Repost with additional data since I can't re-open.)
### Problem / Opportunity
Currently, the import script for book data only queries Amazon US when retrieving metadata. This reflects a US-centric bias that contradicts Open Library’s stated mission:
"Open Library is a universal catalog for book metadata. Our goal is to create a web page for every book ever published."
By limiting metadata retrieval to Amazon US, OL is missing bibliographic data available on other regional Amazon domains.. Expanding support for these regions could be a relatively straightforward way to enhance coverage.
The import logic, as implemented in vendors.py, uses the Amazon Product Advertising API 5.0 https://webservices.amazon.com/paapi5/documentation/. The link to the documentation is in a comment in vendors.py.
Here it describes how to configure the host and region for international access:
https://webservices.amazon.com/paapi5/documentation/common-request-parameters.html#host-and-region
In the current code, the region parameter is hardcoded:
region: str = 'us-east-1'
It does not appear to be dynamically set or overridden based on the ISBN or originating market.
### Proposal
- Parse the ISBN or associated metadata to infer the most likely region of an item.
- Map that region to the appropriate Amazon domain (e.g., amazon.co.uk, amazon.de, etc.).
- Attempt API queries against the corresponding Amazon regional endpoint first.
- Fallback to Amazon US if the regional request fails or no data is found.
### Breakdown
### Related files
Refer to [this map of common Endpoints](https://github.com/internetarchive/openlibrary/wiki/Endpoints):
*
### Requirements Checklist
Checklist of requirements that need to be satisfied in order for this issue to be closed:
* [ ]
### Stakeholders
*
### Instructions for Contributors
* **Before** [creating a new branch](https://github.com/internetarchive/openlibrary/wiki/Git-Cheat-Sheet#making-changes-and-creating-a-pull-request) or pushing up changes to a PR, please first [run these commands](https://github.com/internetarchive/openlibrary/wiki/Git-Cheat-Sheet#working-on-your-branch) to ensure your repository is up to date, as the pre-commit bot may add commits to your PRs upstream.
Contributor guide
Assessment
This issue has not been assessed yet.