internetarchive / internetarchive/openlibrary

Update `sqlite3` to >= 3.35.0 to support `RETURNING`

Open
#8,693 0 comments 0 reactions 1 assignee Claimed by @scottbarnes View on GitHub
Lead: @scottbarnes Priority: 3 Type: Feature Request
Dominant language
Python
Stars
6.7k
Forks
2k
Avg merge
2d 19h
Merged PRs (30d)
138

Description

We cannot currently unit test SQL statements using `RETURNING`, because our unit tests rely on an in-memory instance of SQLite, and the SQLite version in the Docker containers does not support the `RETURNING` clause.

We use `RETURNING` in [one place](https://github.com/internetarchive/openlibrary/blob/c6f537357fc856854635808018c6bf50cc765f1f/openlibrary/core/imports.py#L165-L171), but there are more places this would be nice to use to ensure changes don't happen between a `SELECT` and an `INSERT`. But the more we use this, the more we can't unit test these functions, and the import system more generally.

The version of PostgreSQL Open Library uses supports `RETURNING`, but the SQLite version, `3.34.1`, that [ships with bullseye](https://packages.debian.org/bullseye/sqlite3) does not, as [support was first added in 3.35.0](https://www.sqlite.org/lang_returning.html#overview).

However, [bookworm ships with 3.40.1](https://packages.debian.org/bookworm/sqlite3).

Perhaps we could update to bookworm.

### Stakeholders

@cdrini

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.