MagicStack / MagicStack/asyncpg

No rows returned by fetch() when for DELETE rewritten to UPDATE using rule

Open
#1,173 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8.1k
Forks
468
PR merge metrics
No merged PRs in 30d

Description

* **asyncpg version**: 0.29.0
* **PostgreSQL version**: 16.2 (`postgres:latest` Docker image)
* **Python version**: 3.12.3
* **Platform**: Tested on macOS and Linux
* **Do you use pgbouncer?**: No
* **Did you install asyncpg with pip?**: Yes
* **Can the issue be reproduced under both asyncio and [uvloop](https://github.com/magicstack/uvloop)?**: Yes

Unexpected behaviour: It seems `asyncpg` doesn't return the rows returned by a DELETE query rewritten to an UPDATE query by a rule. Perhaps because it's optimizing (the query status is `DELETE 0`, so perhaps it thinks it doesn't need to return any rows) or something like that? I didn't dive in any further to check if that is indeed what is happening.

Reproduction

```py
import asyncio
import asyncpg

async def main():
connection = await asyncpg.connect('postgresql://postgres:password@localhost/test')

async def fetch_print(query, *params):
result = await connection.fetch(query, *params)
i = 0
for row in result:
print(f' - {row}')
i += 1
if i == 0:
print(' (no rows returned)')
print('')

# Create table with rule for deletion
await connection.execute('''
CREATE TABLE items (
id serial PRIMARY KEY,
name text UNIQUE,
deleted boolean DEFAULT false
);
CREATE RULE softdelete AS
ON DELETE
TO items
DO INSTEAD
UPDATE items SET deleted = true WHERE id = OLD.id RETURNING OLD.*
;
INSERT INTO items (name) VALUES
('foo'),
('bar')
;
''')

print('Our table has a rule that updates the "deleted" column instead of deleting the row.\n')

# Show contents
print('We start with 2 rows which are not soft-deleted:')
await fetch_print('SELECT * FROM items')

# Try delete (the unexpected case)
print('Deleting with RETURNING should give us a row (it does in psql), but we do not get any in asyncpg:')
await fetch_print('''
DELETE FROM items WHERE name = $1 RETURNING id
''', 'foo')

# Confirm above query worked
print('But the row is now soft-deleted:')
await fetch_print('SELECT * FROM items')

# Workaround
print('If wrapped in a CTE it does work:')
await fetch_print('''
WITH x AS (
DELETE FROM items WHERE name = $1 RETURNING id
) SELECT * FROM x
''', 'bar')

# Confirm above query worked
print('And it is again properly softdeleted:')
await fetch_print('SELECT * FROM items')

print('We now delete the rule.\n')
await connection.execute('DROP RULE softdelete ON items')

# Confirm normal delete without rule returns rows
print('Normal deletion (without the rule) does return rows correctly:')
await fetch_print('''
DELETE FROM items RETURNING id
''')

# Confirm above query worked
print('And now both rows are indeed gone:')
await fetch_print('SELECT * FROM items')

# Clean up table after we are done
await connection.execute('DROP TABLE items')

await connection.close()

asyncio.run(main())
```

Output of reproduction

```
Our table has a rule that updates the "deleted" column instead of deleting the row.

We start with 2 rows which are not soft-deleted:
-
-

Deleting with RETURNING should give us a row (it does in psql), but we do not get any in asyncpg:
(no rows returned)

But the row is now soft-deleted:
-
-

If wrapped in a CTE it does work:
-

And it is again properly softdeleted:
-
-

We now delete the rule.

Normal deletion (without the rule) does return rows correctly:
-
-

And now both rows are indeed gone:
(no rows returned)

```

In contrast the `psql` command line tool *does* show me the resulting rows when the result code is `DELETED 0`.

Output of psql

```
test=# DELETE FROM items WHERE name = 'foo' RETURNING id;
id
----
1
(1 row)

DELETE 0
```

So it is a bit unexpected that `asyncpg` doesn't return any rows.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the Python reproduction in the issue against PostgreSQL 16.2, comparing the rule-rewritten DELETE with RETURNING, the CTE workaround, and a normal DELETE. Trace the asyncpg fetch result for the direct rewritten query; done means it returns the row shown by psql while preserving the existing behavior for the other cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.