internetarchive / internetarchive/openlibrary
Bot Account Application: pillarsdotnetBot339
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 2k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 138
Description
## Bot Account Application Request
**Account Name**: `pillarsdotnetBot339` — rename to `pillarsdotnetBot` requested below
**GitHub Username**: pillarsdotnet
**Purpose**: Fill *empty* physical-description fields on editions I own a copy of, from the copy in hand, one human-approved edit at a time.
### Request
- Please grant bot privileges to the `pillarsdotnetBot339` account
- Please add it to the "API" usergroup
- If possible, please **rename it to `pillarsdotnetBot`** (see below) — the bot
convention is that the username ends in `Bot`, and mine currently ends in `339`
### About the name
My first registration used an email address that turned out to be undeliverable
(`pillarsdotnet-bot@gmail.com` — hyphens are not valid in Gmail usernames, so it
bounced and I never got the confirmation). That dead registration still claimed
the `pillarsdotnetBot` screenname on the archive.org side. When I registered
again with a working address, the screenname collided and
`append_random_suffix()` gave me `pillarsdotnetBot339`.
So the name I want is held by an unverified account of my own that can never be
confirmed. I have since changed the archive.org **screen name** to
`pillarsdotnetBot`, but that is only the display name — the account identifier
is unchanged, and my Open Library profile is still
[/people/pillarsdotnetbot339](https://openlibrary.org/people/pillarsdotnetbot339).
If someone can release the orphaned name and rename this account, the bot ends
in `Bot` as the convention intends. If that is more trouble than it is worth, I
am happy to keep `pillarsdotnetbot339` — I would just rather Recent Changes be
able to filter it as a bot.
To be clear about why I am asking, since this is not a bulk import: I am not
requesting `/api/import` access and this bot creates no records. I need
`/usergroup/api` because `can_write()` gates *every* API `PUT`, so even a single
human-reviewed field edit returns 403 without it. If there is a lighter-weight
way to grant that, I would prefer it.
> **Superseded in part — see the [amendment](#amendment-2026-07-21-i-would-also-like-to-import-books-open-library-does-not-have)
> at the end of this issue.** I have since asked to add `/api/import` as well,
> for books Open Library has no edition of. The paragraph above stands as
> originally filed; the amendment says what changed and why.
### Why this matters
Open Library is where my [home-library](https://github.com/pillarsdotnet/library)
app gets its metadata, and it is very good at bibliographic data and thin on
physical description — how tall the book is, how it is bound, how many pages
it actually has. Those are exactly the things you can only know by holding the
book, and I am holding several hundred of them. This is an attempt to pay back
what I take on every ISBN lookup.
### What is queued right now
Rather than describe this in the abstract: the queue has been running against
live Open Library records with the write disabled, which is a shadow window by
another name. As of filing it holds **47 proposals across 19 books** — every one
computed from a real record, none sent:
| Field | Record | Queued |
|---|---|---|
| `physical_dimensions` | edition | 18 |
| `physical_format` | edition | 13 |
| `number_of_pages` | edition | 6 |
| `series:` subject tag | work | 5 |
| cover image | edition | 5 |
Two of each, verbatim, as they would be sent:
| Book | Edition / Work | Field | Value to add |
|---|---|---|---|
| Artemis | `OL28218453M` | `physical_dimensions` | 20.2 x 13.2 x 2.2 centimeters |
| Divine Rivals | `OL38122429M` | `physical_dimensions` | 24.1 x 16.2 x 3.1 centimeters |
| Alchemy's Daughter | `OL34834798M` | `physical_format` | paperback |
| Divine Rivals | `OL38122429M` | `physical_format` | hardback |
| Alchemy's Daughter | `OL34834798M` | `number_of_pages` | 264 |
| How to Become the Dark Lord… | `OL50710754M` | `number_of_pages` | 419 |
| Enchanted | `OL17838987W` | `series` | `series:Woodcutter Sisters` |
| Hero | `OL19969990W` | `series` | `series:Woodcutter Sisters` |
| Alchemy's Daughter | `OL34834798M` | `cover` | my photograph of the copy |
| Enchanted | `OL28492402M` | `cover` | my photograph of the copy |
Note the series rows point at works (`OL…W`) and everything else at editions
(`OL…M`), per the contributors' FAQ on series tags. Happy to post the full list
of 47 if that is more useful than a sample.
### Scope
Five fields, and only ever when Open Library has **nothing** there:
| Field | Record | Source |
|---|---|---|
| `physical_dimensions` | edition | measured to the nearest mm, sent in cm |
| `physical_format` | edition | the binding of the copy in hand (`hardback` / `paperback` only) |
| `number_of_pages` | edition | see convention below |
| `series:` subject tag | work | per the contributors' FAQ on series tags |
| cover image | edition | my own photograph of the copy |
### Guardrails (per edit)
1. `GET` the edition (and the work, for a series tag).
2. Propose **only** fields that are absent — a blank, a zero, an empty array, or
a missing key. **A value already present is never touched**, even where mine
differs. A disagreement is not a correction.
3. The proposal goes into a review queue and waits for **me** to look at it.
4. On approval, **re-read the live record** and abort if the blank has been
filled since it was queued.
5. `PUT` the record back whole, with a factual edit comment.
- **Never overwrites, never deletes, never reorders.** The series tag is
*appended* to `subjects`; existing subjects are preserved untouched.
- **No creation** of works, editions or authors; no merges; no author edits.
- **Digital formats are never sent** as a binding — an ebook is not a binding.
- Partial dimensions are never sent: all three of height/width/thickness, or
nothing.
### Page count convention
This is the field where "missing" and "different" get confused, so it is stated
in the edit comment every time: **the highest explicitly numbered page,
disregarding unnumbered pages.** And because editions honestly differ on what
counts, a page count Open Library already holds is left alone even when mine
disagrees.
### Series position, and why none is sent
The sanctioned `series:Name` tag has nowhere to put a position, which suits me,
because position is contested in the same way page count is — and I would rather
not import my opinion into your catalogue.
Take Travis Baldree's Legends & Lattes books. I shelve them in in-story
chronological order:
1. Bookshops & Bonedust
2. Legends & Lattes
3. Brigands & Breadknives
Plenty of sources order them by publication instead, which swaps the first two.
And Open Library currently holds a third answer — the edition record for
Bookshops & Bonedust carries `series: ["Legends & Lattes, #0"]`, numbering the
prequel zero, while Legends & Lattes itself carries a bare `["Legends & Lattes"]`.
Three defensible orderings, none of them wrong. My reading order is a shelving
preference, not a fact about the book, so it stays on my shelf: the tag names
the series and stops there. (My own lookup code takes a position *from* Open
Library when an edition offers one, but never sends one back — and it reads
`#0` as "no position" rather than as book zero.)
### Edit comments
Verbatim, from the source:
> Add physical dimensions measured from the book.
> Add the binding, taken from the copy in hand.
> Add a page count: the highest explicitly numbered page, disregarding unnumbered pages.
> Add a series tag for this work.
> Add a cover photograph of this edition.
### Volume
Human-paced, because a human approves each one. Realistically a handful of edits
in a sitting, as I work through a shelf — not a batch job, and it has no
unattended mode. I am happy to accept an explicit rate cap as a condition.
### Not an LLM editing Open Library
The values come from the book and from my own database, not from a model. There
is no generation step anywhere in the write path.
### Code
This is a feature of a self-hosted home-library app rather than a standalone
script, which I realise is different from most applications here. The relevant
module is one file with no dependencies —
[`openlibrary.js`](https://github.com/pillarsdotnet/library/blob/main/openlibrary.js) —
covering gap detection, the never-overwrite rule and the send path, with tests
in [`test/openlibrary.test.mjs`](https://github.com/pillarsdotnet/library/blob/main/test/openlibrary.test.mjs).
If you would rather review it as a PR against `openlibrary-bots`, say so and I
will extract it into a standalone script there.
Dry run against a live record (`OL34834798M`, whose `physical_dimensions` is
empty), one edit, nothing else:
```
POST /account/login.json -> 200, session cookie received
GET /books/OL34834798M.json -> 200
PUT /books/OL34834798M.json -> 403 Forbidden
"Permission Denied."
```
The record is unchanged by the attempt — still revision 2, `last_modified`
2023-01-15, `physical_dimensions` still null. The 403 is `can_write()` doing its
job, which is exactly what this request is about: the account is not in
`/usergroup/api` (checked against the public `/usergroup/api.json`), so every
API `PUT` is refused regardless of how small or how well reviewed.
### Operator contact
- GitHub: @pillarsdotnet
- Open Library account: `pillarsdotnetBot`
Happy to narrow the scope, drop any field you would rather not receive this way,
or keep the queue in its current write-disabled state for as long as you would
like to watch what it proposes.
@mekarpeles @hornc
---
### Amendment (2026-07-21): I would also like to import books Open Library does not have
I am flagging a change to my own request rather than quietly widening it later.
**The original text above says I am not requesting `/api/import` access and that
this bot creates no records.** Having looked at my shelf against yours, I would
like to ask for that too — and I would rather raise it now, while nothing has
been granted, than come back for it after approval.
If you would prefer to keep this application to the field-filling scope and
handle imports as a separate request later, say so and I will withdraw this part
without argument.
#### Why
A sample of 60 of my books with ISBNs: **14 have no Open Library edition at
all** — a little under a quarter. There is nothing to contribute to on those,
however good the data. The gap is not random; it is mostly recent technical
publishing:
```
0997671106 The ABCs of RPGs
9781593278939 Absolute FreeBSD, 3rd Edition
9781593275235 Absolute OpenBSD
9781098199173 AI and ML for Coders in PyTorch
9780760376355 The Anime Chef Cookbook
9781593279936 Automate the Boring Stuff with Python
9781718502659 The Book of Kubernetes
9781593276218 The Book of PF
```
These are all books I own, so what I would be adding is first-hand: the record
comes off the copyright page and the physical copy, not from a scrape.
#### How it would work
Same review queue, same one-at-a-time human approval as everything else, plus a
rehearsal step:
1. `GET /isbn/{isbn}.json` — proceed only on a 404. If Open Library has the
edition, this is a field-filling case, not an import.
2. Build the record. Complete where the book allows (title, authors,
publishers, publish_date), otherwise title plus a strong identifier, per
`import_validator`. Physical description is included when known:
`number_of_pages`, `physical_format`, `physical_dimensions`.
3. **`POST /api/import?preview=true`** — the rehearsal. Your own matching runs
and nothing is saved.
4. If the preview reports `edition.status == "matched"`, **stop**: the book
already exists under another identifier and the queue records which record it
matched. Only a preview that would genuinely create something goes on to
5. `POST /api/import` for real, one book, on my explicit approval.
Deduplication is decided by your matcher, not by my code. A duplicate that a
librarian then has to merge is the failure mode I am most trying to avoid, since
I could not clean it up myself.
The feature is **switched off in the code** (`OPENLIBRARY_ALLOW_IMPORT`, unset by
default) precisely because the application above says it creates nothing. It
stays off until you tell me otherwise.
#### The question I actually need answered
**What `source_records` prefix should I use?**
`import_validator` requires a non-empty `source_records`, and the prefix
identifies where a record came from. I have deliberately not invented one —
making up a provenance string seems like exactly the wrong thing to put in a
public catalogue. If there is an existing convention for a personal-library
contributor, I will use it; otherwise I am happy to be assigned something like
`pillarsdotnetbot:`.
I need this from you whether or not the import scope is granted today, so it is
worth asking regardless.
Contributor guide
Research direction
Review the requested /usergroup/api permission and the amended /api/import flow against the Open Library API behavior described in the issue. Read openlibrary.js and test/openlibrary.test.mjs for the proposed client-side safeguards, then determine whether the account request fits existing policy and permission mechanisms; done means a maintainer-approved scope and access decision.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, python
- Domain
- api, authorization, backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100