internetarchive / internetarchive/openlibrary

feat(imports): Allow specifying a batch name when submitting to /import/batch/new

Open
#12,656 1 comment 0 reactions 0 assignees View on GitHub
Lead: @mekarpeles Module: Import Needs: Staff Decision Priority: 2 Type: Feature Request Type: Subtask of Epic
Dominant language
Python
Stars
6.7k
Forks
2k
Avg merge
2d 19h
Merged PRs (30d)
138

Description

### Problem

`POST /import/batch/new` always generates a batch name from a SHA-256 hash of the submitted payload. There is no way to specify a name, so every submission — including retries and chunked imports — creates a new batch instead of adding to an existing one.

The server-side primitive already supports named batches correctly (see [`partner_batch_imports.py`](https://github.com/internetarchive/openlibrary/blob/master/scripts/partner_batch_imports.py#L423)):

```python
batch = Batch.find(batch_name) or Batch.new(batch_name, submitter=username)
batch.add_items(batch_data)
```

It just isn't exposed through the endpoint.

### Proposed Change

Accept an optional `batchName` field in the `POST /import/batch/new` form data:

- If provided, use `Batch.find(batch_name) or Batch.new(batch_name)` — adding records to the existing batch if it exists
- If absent, fall back to the current hash-derived name

### Relevant files

- `openlibrary/core/batch_imports.py` — `batch_import()` — where the hash name is generated
- `openlibrary/plugins/openlibrary/code.py` — `batch_imports.POST()` — where form data is read
- `openlibrary/core/imports.py` — `Batch.find()` / `Batch.new()` — already support named lookup

### Related

- Wiki: https://github.com/internetarchive/openlibrary/wiki/The-Import-Pipeline#Import-Queue (lists "Adding to an existing batch" as a planned agenda item)
- Epic: #12655

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.