MagicStack / MagicStack/asyncpg

List[str] parameter treated as text instead of text[] in certain requests

Open
#996 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8.1k
Forks
468
PR merge metrics
No merged PRs in 30d

Description

* **asyncpg version**: 0.27.0
* **PostgreSQL version**: 15.1
* **Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
the issue with a local PostgreSQL install?**: No
* **Python version**: 3.8.10
* **Platform**: Ubuntu
* **Do you use pgbouncer?**: No
* **Did you install asyncpg with pip?**: Yes
* **If you built asyncpg locally, which version of Cython did you use?**:
* **Can the issue be reproduced under both asyncio and
[uvloop](https://github.com/magicstack/uvloop)?**: Yes

I'm working with a custom bulk operations API built on top of SQLAlchemy and asyncpg, and some scenarios the List parameters are not treated as postgres arrays.

Example 1:
```
import asyncio
import asyncpg

async def try_it(table: str, query: str, *params, **connargs):
conn = await asyncpg.connect(**connargs)
try:
await conn.execute(f'CREATE TABLE test_table({table});')
await conn.execute(query, *params)
finally:
await conn.execute('DROP TABLE test_table;')
await conn.close()

table = 'a text[], b text'
query = """
UPDATE test_table SET a = uvals.a
FROM (VALUES ($1, $2)) AS uvals (a, b)
WHERE test_table.b = uvals.b
"""
params = [ ['hello', 'world'], 'helloworld']

db_conn_params = {}
asyncio.get_event_loop().run_until_complete(try_it(table, query, *params, **db_conn_params))
```

Response:
```
asyncpg.exceptions.DatatypeMismatchError: column "a" is of type text[] but expression is of type text
```

Example 2:
```
# same imports and try_it() from above

table = 'a text, b int'
query = """
SELECT a, b
FROM test_table
UNION
SELECT
values as a,
5 as b
FROM unnest($1) as values
"""
params = [ ['hello', 'world'] ]

# execution as above
```

Response:
```
asyncpg.exceptions.AmbiguousFunctionError: function unnest(unknown) is not unique
```

Adding a `$1 :: text[]` solves the problem in each case, since it appears to be passing the list as text but there are scenarios where I don't have direct control of the SQL (it being auto-generated).

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Run the two try_it() reproductions against asyncpg 0.27.0 and PostgreSQL 15.1, comparing them with the explicit ::text[] casts. Trace how List parameters are inferred and encoded for VALUES and unnest queries. Done means both examples treat the list as text[] without requiring SQL casts, with regression coverage for the reported cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.