internetarchive / internetarchive/openlibrary

Epic: Lenny ↔ Open Library integration

Open
#13,270 1 comment 0 reactions 0 assignees View on GitHub
Lead: @mekarpeles Needs: Breakdown Priority: 2 Type: Epic Type: Feature Request
Dominant language
Python
Stars
6.7k
Forks
2k
Avg merge
2d 14h
Merged PRs (30d)
126

Description

# Epic: Lenny ↔ Open Library integration

Filed by @openlibrary-bot on behalf of @mekarpeles, capturing the integration plan and the verified state of each piece as of 2026-08-05.

## Target Deadline

End of September, 2027 (for October celebration)

## Why this is its own epic

Open Library already links to books from several **Trusted Book Providers** — mission-aligned organizations like Standard Ebooks and Project Gutenberg. **Lenny is the first case where a service other than archive.org will be *lending* books.**

Technically, [lennyforlibraries](https://github.com/ArchiveLabs/lenny) is being run as an experimental Labs node on behalf of the Internet Archive, lending out the born-digital books it has purchased through BRIET. But it is still new terrain: every existing assumption in Open Library about "who can hold a loan" and "where a patron's loans live" currently resolves to archive.org. This epic tracks making a second lending catalog a first-class citizen.

Existing issues each cover one slice of this and none covers the whole; this epic is the umbrella, not a replacement:

- #11384 — Index Lenny instance(s) holdings in Open Library (holdings sync / `lenny_host` in Solr)
- #12844 — Feed Registry & Acquisitions Import System (Epic) (provider-agnostic feed registry + acquisitions)
- #12138 — Open Library Solr Bulk Indexing BWB & Lenny OPDS (price)
- #5792 — Trusted Book Providers (epic)
- #11264 — Add Solr acquisitions field for BookServer integration

## Plan

- [x] **1. Spin up a Lenny node** (code needs to be updated / pulled)
- [ ] **2. Register the Lenny OPDS feed via #12844**, so Open Library acquisitions and Solr are up to date with Lenny being a lending option for these titles
- [ ] **3. Audit why Lenny OPDS feeds are so slow**, and whether they can be improved/optimized — tracked upstream at ArchiveLabs/lenny#194
- [ ] **4. Add and integration-test borrowing a Lenny book from OpenLibrary.org** — a logged-in patron clicks Borrow on OL, which makes an AJAX POST to Lenny with OL S3 keys for auth, resulting in success and possibly a cookie telling OL this patron may have loans on Lenny
- [ ] **5. When fetching a patron's loans, check cookies** to see whether there are other catalogs (i.e. Lenny) that OL should also be fetching that patron's loans from

## Verified state of each item (2026-08-05)

**1 — Lenny node.** Done. Note the local checkout drifts quickly: `ArchiveLabs/lenny` `main` is at `50d5c0b` (Merge PR #191). Pull before testing anything against it.

**2 — Feed registration.** Blocked on #12844's Subtask 1: the `tbp_feed_registry` table was designed in PR #12846, but that PR is **closed, unmerged, and now conflicting** — it needs a fresh PR rebased on current master before a Lenny feed row can be registered at all. The `acquisitions` table (Subtask 2) **is** merged via PR #12851, but nothing in the live import/acquisition flow populates or reads it yet. The BWB OPDS importer (PR #12852, open draft, @ronibhakta1) is the template the Lenny feed would follow; generalizing that connector to handle per-feed identifier schemes and cursor query params is the concrete next piece of work.

**3 — OPDS performance.** Not started. Tracking issue opened upstream: ArchiveLabs/lenny#194. No prior performance issue existed on that repo.

**4 — Borrow from OL.** **The Lenny side of this already exists and is merged.** `IA_AUTH_ENABLED` gates an optional IA-auth plugin: a patron sends `Authorization: LOW :` to `POST /v1/api/items/{id}/borrow`, Lenny validates the S3 keys against `archive.org/services/xauthn/?op=s3auth`, extracts the IA screenname as the patron identifier, creates the loan record, and sets a signed session cookie. See `lenny/core/patron_auth/manager.py`, `lenny/routes/api.py`, `tests/test_ia_s3_auth.py`.

History worth recording, because it is confusing from the outside: ArchiveLabs/lenny#183 (issue) and #184 (PR) are both **closed and unmerged** — #184 was superseded by #185, which **merged 2026-06-10**. The stale branch `183/ia-patron-auth` still exists on the remote, one commit ahead of an old `main`; it is not the source of truth.

So item 4's remaining work is entirely **Open Library side**: the Borrow button, the AJAX POST carrying OL S3 keys, the success/failure UX, the cookie handshake, and integration tests across the boundary.

**5 — Cross-catalog loans.** Not started. Depends on the cookie contract established in item 4, so those two should be designed together rather than sequentially.

## Open questions

- What exactly does the cookie set by Lenny in item 4 contain, and is a cookie the right mechanism for item 5 — or should OL persist "this patron has a Lenny account" server-side? A cookie is lost on device change, which would silently hide a patron's active loans on another browser.
- #12844 already flags an unresolved scope overlap between itself, #5792, and #12655 (BookWorm). Item 2 lands squarely inside that overlap; worth resolving before more subtask PRs land.
- Does a Lenny loan need to appear in the patron's OL loan count / borrow limits, or is it accounted separately? This affects item 5's design well beyond fetching.

### Stakeholders

@mekarpeles @cdrini @ronibhakta1

Contributor guide

Open the contributing guide

Research direction

Start by reading the linked component issues, especially #12844 and #11384, to understand the feed and acquisitions dependencies. For the borrowing boundary, inspect lenny/core/patron_auth/manager.py, lenny/routes/api.py, and tests/test_ia_s3_auth.py, then identify the corresponding Open Library borrowing and loan-fetch entry points. Done means the agreed feed, borrowing, cookie, and cross-catalog loan behavior is implemented and integration-tested.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.