internetarchive / internetarchive/openlibrary

Upgrade to psycopg3 for performance increases

Open
#10,258 0 comments 0 reactions 0 assignees View on GitHub
Affects: Database Lead: @mekarpeles Priority: 3 Theme: Performance Type: Feature Request
Dominant language
Python
Stars
6.7k
Forks
2k
Avg merge
2d 19h
Merged PRs (30d)
138

Description

### Proposal

Related to https://github.com/internetarchive/openlibrary/issues/6052
Also related to the performance discussion https://internetarchive.slack.com/archives/C0ETZV72L/p1735828712694179

A relevant [reddit thread](https://www.reddit.com/r/learnpython/comments/vi59dk/aiopg_vs_asyncpg_vs_psycopg3/):
> if you are using psycopg2 already and want to go async, it should be straightforward to upgrade to psycopg3

Impressive benchmarks:
- https://www.timescale.com/blog/psycopg2-vs-psycopg3-performance-benchmark
- https://www.spherex.dev/psycopg-2-vs-psycopg3/

What's different:
https://www.psycopg.org/psycopg3/docs/basic/from_pg2.html

No longer blocked! by #5675

If you want to work on this please look at this old PR https://github.com/internetarchive/openlibrary/pull/10259
Also https://github.com/webpy/webpy/pull/606

### Justification

_No response_

### Breakdown

#### Requirements Checklist

* [ ]

#### Related files

*

#### Stakeholders

*


#### Instructions for Contributors

Please [run these commands](https://github.com/internetarchive/openlibrary/wiki/Git-Cheat-Sheet#working-on-your-branch) to ensure your repository is up to date **before** [creating a new branch](https://github.com/internetarchive/openlibrary/wiki/Git-Cheat-Sheet#making-changes-and-creating-a-pull-request) to work on this issue and **each time after** pushing code to Github, because the pre-commit bot may add commits to your PRs upstream.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.