internetarchive / internetarchive/openlibrary
Epic: Lenny ↔ Open Library integration
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 2k
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 126
Description
# Epic: Lenny ↔ Open Library integration
Filed by @openlibrary-bot on behalf of @mekarpeles, capturing the integration plan and the verified state of each piece as of 2026-08-05.
## Target Deadline
End of September, 2027 (for October celebration)
## Why this is its own epic
Open Library already links to books from several **Trusted Book Providers** — mission-aligned organizations like Standard Ebooks and Project Gutenberg. **Lenny is the first case where a service other than archive.org will be *lending* books.**
Technically, [lennyforlibraries](https://github.com/ArchiveLabs/lenny) is being run as an experimental Labs node on behalf of the Internet Archive, lending out the born-digital books it has purchased through BRIET. But it is still new terrain: every existing assumption in Open Library about "who can hold a loan" and "where a patron's loans live" currently resolves to archive.org. This epic tracks making a second lending catalog a first-class citizen.
Existing issues each cover one slice of this and none covers the whole; this epic is the umbrella, not a replacement:
- #11384 — Index Lenny instance(s) holdings in Open Library (holdings sync / `lenny_host` in Solr)
- #12844 — Feed Registry & Acquisitions Import System (Epic) (provider-agnostic feed registry + acquisitions)
- #12138 — Open Library Solr Bulk Indexing BWB & Lenny OPDS (price)
- #5792 — Trusted Book Providers (epic)
- #11264 — Add Solr acquisitions field for BookServer integration
## Plan
- [x] **1. Spin up a Lenny node** (code needs to be updated / pulled)
- [ ] **2. Register the Lenny OPDS feed via #12844**, so Open Library acquisitions and Solr are up to date with Lenny being a lending option for these titles
- [ ] **3. Audit why Lenny OPDS feeds are so slow**, and whether they can be improved/optimized — tracked upstream at ArchiveLabs/lenny#194
- [ ] **4. Add and integration-test borrowing a Lenny book from OpenLibrary.org** — a logged-in patron clicks Borrow on OL, which makes an AJAX POST to Lenny with OL S3 keys for auth, resulting in success and possibly a cookie telling OL this patron may have loans on Lenny
- [ ] **5. When fetching a patron's loans, check cookies** to see whether there are other catalogs (i.e. Lenny) that OL should also be fetching that patron's loans from
## Verified state of each item (2026-08-05)
**1 — Lenny node.** Done. Note the local checkout drifts quickly: `ArchiveLabs/lenny` `main` is at `50d5c0b` (Merge PR #191). Pull before testing anything against it.
**2 — Feed registration.** Blocked on #12844's Subtask 1: the `tbp_feed_registry` table was designed in PR #12846, but that PR is **closed, unmerged, and now conflicting** — it needs a fresh PR rebased on current master before a Lenny feed row can be registered at all. The `acquisitions` table (Subtask 2) **is** merged via PR #12851, but nothing in the live import/acquisition flow populates or reads it yet. The BWB OPDS importer (PR #12852, open draft, @ronibhakta1) is the template the Lenny feed would follow; generalizing that connector to handle per-feed identifier schemes and cursor query params is the concrete next piece of work.
**3 — OPDS performance.** Not started. Tracking issue opened upstream: ArchiveLabs/lenny#194. No prior performance issue existed on that repo.
**4 — Borrow from OL.** **The Lenny side of this already exists and is merged.** `IA_AUTH_ENABLED` gates an optional IA-auth plugin: a patron sends `Authorization: LOW :` to `POST /v1/api/items/{id}/borrow`, Lenny validates the S3 keys against `archive.org/services/xauthn/?op=s3auth`, extracts the IA screenname as the patron identifier, creates the loan record, and sets a signed session cookie. See `lenny/core/patron_auth/manager.py`, `lenny/routes/api.py`, `tests/test_ia_s3_auth.py`.
History worth recording, because it is confusing from the outside: ArchiveLabs/lenny#183 (issue) and #184 (PR) are both **closed and unmerged** — #184 was superseded by #185, which **merged 2026-06-10**. The stale branch `183/ia-patron-auth` still exists on the remote, one commit ahead of an old `main`; it is not the source of truth.
So item 4's remaining work is entirely **Open Library side**: the Borrow button, the AJAX POST carrying OL S3 keys, the success/failure UX, the cookie handshake, and integration tests across the boundary.
**5 — Cross-catalog loans.** Not started. Depends on the cookie contract established in item 4, so those two should be designed together rather than sequentially.
## Open questions
- What exactly does the cookie set by Lenny in item 4 contain, and is a cookie the right mechanism for item 5 — or should OL persist "this patron has a Lenny account" server-side? A cookie is lost on device change, which would silently hide a patron's active loans on another browser.
- #12844 already flags an unresolved scope overlap between itself, #5792, and #12655 (BookWorm). Item 2 lands squarely inside that overlap; worth resolving before more subtask PRs land.
- Does a Lenny loan need to appear in the patron's OL loan count / borrow limits, or is it accounted separately? This affects item 5's design well beyond fetching.
### Stakeholders
@mekarpeles @cdrini @ronibhakta1
Contributor guide
Research direction
Start by reading the linked component issues, especially #12844 and #11384, to understand the feed and acquisitions dependencies. For the borrowing boundary, inspect lenny/core/patron_auth/manager.py, lenny/routes/api.py, and tests/test_ia_s3_auth.py, then identify the corresponding Open Library borrowing and loan-fetch entry points. Done means the agreed feed, borrowing, cookie, and cross-catalog loan behavior is implemented and integration-tested.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend-api-design, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100