internetarchive / internetarchive/openlibrary

Check other regions when importing from Amazon, not just US

Open
#10,707 2 comments 1 reaction 0 assignees View on GitHub
Lead: @scottbarnes Needs: Breakdown Needs: Staff / Internal Priority: 3 Type: Feature Request
Dominant language
Python
Stars
6.7k
Forks
2k
Avg merge
2d 19h
Merged PRs (30d)
138

Description

### Feature Request

(Repost with additional data since I can't re-open.)

### Problem / Opportunity

Currently, the import script for book data only queries Amazon US when retrieving metadata. This reflects a US-centric bias that contradicts Open Library’s stated mission:

"Open Library is a universal catalog for book metadata. Our goal is to create a web page for every book ever published."

By limiting metadata retrieval to Amazon US, OL is missing bibliographic data available on other regional Amazon domains.. Expanding support for these regions could be a relatively straightforward way to enhance coverage.

The import logic, as implemented in vendors.py, uses the Amazon Product Advertising API 5.0 https://webservices.amazon.com/paapi5/documentation/. The link to the documentation is in a comment in vendors.py.
Here it describes how to configure the host and region for international access:

https://webservices.amazon.com/paapi5/documentation/common-request-parameters.html#host-and-region

In the current code, the region parameter is hardcoded:

region: str = 'us-east-1'

It does not appear to be dynamically set or overridden based on the ISBN or originating market.

### Proposal

- Parse the ISBN or associated metadata to infer the most likely region of an item.

- Map that region to the appropriate Amazon domain (e.g., amazon.co.uk, amazon.de, etc.).

- Attempt API queries against the corresponding Amazon regional endpoint first.

- Fallback to Amazon US if the regional request fails or no data is found.

### Breakdown

### Related files

Refer to [this map of common Endpoints](https://github.com/internetarchive/openlibrary/wiki/Endpoints):
*

### Requirements Checklist
Checklist of requirements that need to be satisfied in order for this issue to be closed:
* [ ]

### Stakeholders

*


### Instructions for Contributors

* **Before** [creating a new branch](https://github.com/internetarchive/openlibrary/wiki/Git-Cheat-Sheet#making-changes-and-creating-a-pull-request) or pushing up changes to a PR, please first [run these commands](https://github.com/internetarchive/openlibrary/wiki/Git-Cheat-Sheet#working-on-your-branch) to ensure your repository is up to date, as the pre-commit bot may add commits to your PRs upstream.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.