python / python/cpython

sqlite3: wrap sqlite3_txn_state and sqlite3_stmt_readonly C functions

オープン
#141,463 コメント 1 件 リアクション 0 件 担当者 0 名 GitHub で見る

まだ誰も着手していません。

extension-modules topic-sqlite3 type-feature
主要言語
Python
スター
77.2k
フォーク
35.9k
PR マージ指標
PR 指標を取得中

説明

Feature or enhancement

Proposal:

SQLite is well known for having locking problems, as it's serverless. WAL mode reduces these but does not eliminate them. Debugging SQLITE_BUSY errors is hard since they can be triggered by any Cursor/Connection execute method call, anywhere in the codebase. It can be difficult to identify the blocking process without instrumenting every one of these calls, since it's not obvious from higher up whether a particular query is read-only or read-write, or what locks are already held.

It would be helpful to be able to identify whether our own connection currently has a read lock (SHARED) or a write lock (EXCLUSIVE) and whether it's about to (try to) acquire one. This can be done using the C API functions:

These functions are not wrapped and not available in Python, not even using ctypes since we can't get access to the raw sqlite3 structure pointer/handle from Python.

I'm imagining being able to use a custom Cursor class to do something like this:

  • In overridden execute() method
  • If we are about to execute a non-readonly statement
  • And we are not already in a write transaction
  • Check for another PID in a shared state file (e.g. a lock-type file)
  • If there is one, our own attempt to acquire the lock may be delayed or fail, so log that
  • Attempt to acquire a write lock with BEGIN IMMEDIATE
  • If it was delayed, log the delay
  • If it fails, log the PID that was holding the lock before and reraise
  • Write our own PID to the shared state file
  • Execute the non-readonly statement (this should not now block)
  • If we were not in a transaction before, then COMMIT (this may also block as a checkpoint could require exclusive access/no readers) and remove the shared state file

I might be able to submit a PR for this as it should not be too difficult to wrap these C functions. I've been able to use sqlite3_txn_state by creating and loading a custom SQLite3 extension (in C) that creates a custom function that calls this function.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

sqlite3_stmt_readonly used to be used internally by Cursor for a minor feature (the row count indication), but not exposed to Python.

コントリビューションガイド

コントリビューションガイドを開く

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

調査の方向性

Cursor と Connection の実行処理周辺にある CPython の sqlite3 バインディングを特定し、issue 73541 で参照されている以前の内部利用を確認します。sqlite3_txn_state と sqlite3_stmt_readonly を Python に公開する方法を確認し、関連するテストを特定します。両方の SQLite C 関数に Python からアクセスでき、その動作がテストでカバーされれば完了です。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
python, sqlite
領域
databases
issue の種類
機能追加
難易度
3/5
見積もり時間
1〜2日
活発さ
停滞
明瞭さ
おおむね明確
初心者へのやさしさ
42/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。