MagicStack / MagicStack/asyncpg

Regression in v0.31.0: with `statement_cache_size=0`, type introspection reuses the unnamed statement name `""` and clobbers a user statement on the server-side cursor path

Aperta
#1,335 0 commenti 3 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Lingua principale
Python
Stelle
8.1k
Fork
468
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

## Summary

Since **0.31.0**, connecting with `statement_cache_size=0` and then opening a **server-side cursor** over a result that contains an **unknown result type** (e.g. an enum) fails deterministically with:

```
asyncpg.exceptions.ProtocolViolationError:
bind message supplies 2 parameters, but prepared statement "" requires 1
```

The cause is that asyncpg's own type-introspection query is executed as unnamed statement `""`, replacing the user statement prepared as `""` because the cache is disabled. The subsequent cursor bind then targets the wrong statement.

This is a regression from PR #1245 (shipped in 0.31.0), which made `prepare(name=None)` produce the *unnamed* statement `""` when the statement cache is disabled.

Before 0.31.0, `prepare(name=None)` produced a uniquely *named* statement regardless of the cache setting, so it could never collide with the unnamed introspection query.

## Environment

| | |
|---|---|
| asyncpg | 0.31.0 |
| PostgreSQL server | 16.14 |
| Python | 3.14.6 |
| Connection pooler | **none** (direct connection) |
| ORM / driver wrapper | **none** (pure asyncpg) |

## Reproduction

Full setup here:
https://gist.github.com/quentinverlhac/dbbf98401721239f77b969645b9d880d

Requires only asyncpg and a PostgreSQL instance. Point `DSN` at any database where the connecting role may create a type and a table.

```python
"""Pure-asyncpg reproducer of the 0.31.0 statement_cache_size=0 cursor regression.

The failing server-side cursor sequence is:

prepared_stmt = await conn.prepare(operation, name=None)
cursor = await prepared_stmt.cursor(*parameters)

Key conditions to trigger the bug:
- statement_cache_size=0 -> prepare(name=None) produces the UNNAMED "" statement
- result contains an enum -> asyncpg runs its own introspection as "" and clobbers it
- a FRESH connection -> the enum is unknown, so introspection fires here
- >=1 bind params (2 here) -> gives the clean ProtocolViolationError (2 vs 1)

The trigger is asyncpg's own type introspection running *inside* conn.prepare()
on a fresh connection (the enum result type is unknown), which clobbers the ""
statement; the later cursor bind then targets it. This is a direct connection,
no connection pooler involved.
"""
import asyncio
import asyncpg

DSN = dict(user='asyncpg', password='asyncpg_pw',
database='asyncpg_db', host='127.0.0.1', port=5433)

OPERATION = 'SELECT id, value FROM mytable WHERE id > $1 AND id < $2 ORDER BY id'
PARAMETERS = (0, 100)

async def setup():
# Setup on a SEPARATE connection so the streaming connection below is fresh
# and has never introspected my_enum.
conn = await asyncpg.connect(**DSN)
await conn.execute('DROP TABLE IF EXISTS mytable')
await conn.execute('DROP TYPE IF EXISTS my_enum')
await conn.execute("CREATE TYPE my_enum AS ENUM ('value1', 'value2')")
await conn.execute(
'CREATE TABLE mytable(id serial PRIMARY KEY, value my_enum NOT NULL)')
await conn.execute("INSERT INTO mytable(value) VALUES('value1'), ('value2')")
await conn.close()

async def stream_over_cursor():
# Fresh connection with the cache disabled -> unnamed "" statements.
conn = await asyncpg.connect(**DSN, statement_cache_size=0)

# A server-side cursor must run inside a transaction.
tr = conn.transaction()
await tr.start()
try:
# Step 1: prepare(name=None). With cache disabled this is the "" statement.
# Because the result has an unknown enum type, asyncpg's own type
# introspection runs here as "" too, clobbering this user statement.
prepared_stmt = await conn.prepare(OPERATION, name=None)
print('prepared name =', repr(prepared_stmt.get_name()))

# Step 2: drive the server-side cursor with the bound params. The bind now
# lands on the wrong "" statement (the 1-param introspection query).
cursor = await prepared_stmt.cursor(*PARAMETERS)
rows = await cursor.fetch(50)
print('rows =', rows)
finally:
await tr.rollback()
await conn.close()

async def main():
await setup()
await stream_over_cursor()

asyncio.run(main())
```

### Actual result on 0.31.0

```
prepared name = ''
Traceback (most recent call last):
...
File ".../asyncpg/cursor.py", line 144, in _bind
buffer = await protocol.bind(self._state, self._args, ...)
File "asyncpg/protocol/protocol.pyx", line 294, in bind
asyncpg.exceptions.ProtocolViolationError:
bind message supplies 2 parameters, but prepared statement "" requires 1
```

The `prepared name = ''` line confirms the precondition (the user statement is the unnamed `""`), and the `ProtocolViolationError` is the regression.

### Expected result

The cursor should stream the two rows, e.g. `rows = [, ]`.

### Result on 0.30.0

```
prepared name = '__asyncpg_stmt_1__'
rows = [, ]
```

Confirms the regression.

## Server-side wire sequence (from `log_statement='all'`)

The streaming connection, inside its transaction:

```
BEGIN;
execute : -- user SELECT parsed as ""
execute : WITH RECURSIVE typeinfo_tree -- asyncpg enum introspection, ALSO "" (clobbers the SELECT)
execute : -- bind user SELECT (2 params) against "" (now the 1-param introspection)
ERROR: bind message supplies 2 parameters, but prepared statement "" requires 1
```

Both the user statement and asyncpg's internal introspection use the empty statement name `""`, and the introspection runs between prepare and bind.

## Root cause

Since PR #1245, `conn.prepare(operation, name=None)` resolves `name=None` against `self._stmt_cache_enabled`, which is `False` when `statement_cache_size == 0`, so the statement is created **unnamed** (`""`), which is intentional.

However, asyncpg's internal type-introspection query is also unnamed (`""`). When the result contains an unknown type, introspection runs while the user's `""` statement is live and overwrites it.

Later, the cursor bind then targets the introspection statement instead of the initial user statement.

## Notes

- The plain `execute`/`fetch` path is protected by the
`mark_unprepared()` reprepare logic, so it does not suffer from the same issue than the `PreparedStatement.cursor()` bind path.

- This was originally surfaced to me through SQLAlchemy's [stream_scalars](https://github.com/sqlalchemy/sqlalchemy/blob/main/lib/sqlalchemy/ext/asyncio/engine.py#L786) with `asyncpg` dialect and `connect_args={"statement_cache_size": 0}`).
The repro above removes SQLAlchemy entirely and fails identically, confirming the behavior is at the asyncpg level.

## Related PRs

- #1245
- #1243

## Related issues

- #1219
- #1078

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Direzione di ricerca

Inizia da asyncpg/cursor.py:_bind e asyncpg/protocol/protocol.pyx:294, quindi confronta il comportamento di prepare introdotto da PR #1245 con la sequenza di introspezione dei tipi descritta nel report. Il lavoro è completato quando una nuova connessione PostgreSQL con statement_cache_size=0 può preparare e trasmettere la query enum tramite un cursore lato server senza che l’istruzione senza nome venga sovrascritta, preservando il comportamento di 0.30.0.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
postgresql, python
Ambito
databases
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Tranquilla
Chiarezza
Abbastanza chiara
Idoneità per principianti
55/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.