internetarchive / internetarchive/openlibrary

Add geo-awareness into Availability API requests

Open
#10,233 0 comments 0 reactions 0 assignees View on GitHub
Affects: Operations Lead: @mekarpeles Module: Borrowing / Lending Needs: Breakdown Needs: Staff / Internal Priority: 3 State: Blocked Theme: Book Availability Type: Feature Request
Dominant language
Python
Stars
6.7k
Forks
2k
Avg merge
2d 19h
Merged PRs (30d)
138

Description

### Proposal

Requests from Open Library (already attaching `x-preferred-client-ip` headers) will need to include `x-preferred-client-useragent` and `x-preferred-client-jurisdiction` (two-character geoip location country code) along with the request.

This issue can be resolved once:
* [ ] Open Library updated nginx to deduce the jurisdiction from an IP at nginx level, related to #10139
* [ ] core.lending code passes this jurisdiction along to the bulk availability service https://github.com/internetarchive/openlibrary/blob/master/openlibrary/core/lending.py#L378-L400
* [ ] @ximm and @mekarpeles coordinate to update the bulk availability API to pass along jurisdiction to ES

### Justification

Access to some items within the Internet Archive's purview are geographically restricted.

Currently, this only affects a few hundred Open Library records:
https://archive.org/advancedsearch.php?q=collection:geo_restricted%20AND%20openlibrary_edition:*&rows=10&output=json&fl=identifier,geo_restricted,collection&scope=all

### Breakdown

#### Related files

https://github.com/internetarchive/openlibrary/blob/master/openlibrary/core/lending.py#L378-L400

Relates to https://webarchive.jira.com/browse/WEBDEV-7187

#### Stakeholders

* @ximm @cdrini

Contributor guide

Open the contributing guide

Research direction

Start with openlibrary/core/lending.py lines 378-400 and trace how the Availability request is assembled. Coordinate the nginx jurisdiction lookup and the bulk availability API handoff; the work is done when the three checklist items are complete and jurisdiction reaches ES with the requested client headers.

Written by the indexing model from the issue text.

Assessment

Tech stack
nginx, python
Domain
api, backend, search
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.