MagicStack / MagicStack/asyncpg

Unimplemented feature for INSERT queries on ARRAY columns with CockroachDB

Open
#568 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8.1k
Forks
468
PR merge metrics
No merged PRs in 30d

Description

* **asyncpg version**: 0.20.1
* **CockroachDB version**: 19.2.6
* **Python version**: 3.7.4
* **Platform**: MacOS Mojave 10.14.6
* **Do you use pgbouncer?**: No
* **Did you install asyncpg with pip?**: No, with `poetry`
* **Can the issue be reproduced under both asyncio and
[uvloop](https://github.com/magicstack/uvloop)?**: Yes

I'm using `Gino` as a PostgreSQL driver for `CockroachDB`. It doesn't work well for `INSERT` queries with `ARRAY` columns. There seems to be an introspection query executed by asyncpg when it didn't get enough knowledge about the types in the query:

https://github.com/MagicStack/asyncpg/blob/v0.20.1/asyncpg/introspection.py#L99-L134
https://github.com/MagicStack/asyncpg/blob/v0.20.1/asyncpg/connection.py#L328-L338

```
INFO: INSERT INTO image (id, s3_bucket, s3_key, formats, sizes, created_at) VALUES ($1, $2, $3, $4, $5, $6) RETURNING image.id, image.s3_bucket, image.s3_key, image.formats, image.sizes, image.created_at
INFO: ('2941f3a0-e2ea-4d9d-8e23-2ce7b793bcac', 'testbucket', 'iqEb1mDimiUpRsmG', ['jpeg'], [], 1588471074)
INFO: COMMIT
ERROR: Exception in ASGI application
Traceback (most recent call last):
...
created_obj = await cls(**kwargs).create()
File "/Users/son/Library/Caches/pypoetry/virtualenvs/project-rain-7LNSaARJ-py3.7/lib/python3.7/site-packages/gino/crud.py", line 485, in _create
row = await bind.first(q)
File "/Users/son/Library/Caches/pypoetry/virtualenvs/project-rain-7LNSaARJ-py3.7/lib/python3.7/site-packages/gino/engine.py", line 748, in first
return await conn.first(clause, *multiparams, **params)
File "/Users/son/Library/Caches/pypoetry/virtualenvs/project-rain-7LNSaARJ-py3.7/lib/python3.7/site-packages/gino/engine.py", line 328, in first
return await result.execute(one=True)
File "/Users/son/Library/Caches/pypoetry/virtualenvs/project-rain-7LNSaARJ-py3.7/lib/python3.7/site-packages/gino/dialects/base.py", line 215, in execute
context.statement, context.timeout, args, 1 if one else 0
File "/Users/son/Library/Caches/pypoetry/virtualenvs/project-rain-7LNSaARJ-py3.7/lib/python3.7/site-packages/gino/dialects/asyncpg.py", line 184, in async_execute
result, stmt = await getattr(conn, "_do_execute")(query, executor, timeout)
File "/Users/son/Library/Caches/pypoetry/virtualenvs/project-rain-7LNSaARJ-py3.7/lib/python3.7/site-packages/asyncpg/connection.py", line 1423, in _do_execute
stmt = await self._get_statement(query, None)
File "/Users/son/Library/Caches/pypoetry/virtualenvs/project-rain-7LNSaARJ-py3.7/lib/python3.7/site-packages/asyncpg/connection.py", line 338, in _get_statement
types_with_missing_codecs, timeout)
File "/Users/son/Library/Caches/pypoetry/virtualenvs/project-rain-7LNSaARJ-py3.7/lib/python3.7/site-packages/asyncpg/connection.py", line 378, in _introspect_types
self._intro_query, (list(typeoids),), 0, timeout)
File "/Users/son/Library/Caches/pypoetry/virtualenvs/project-rain-7LNSaARJ-py3.7/lib/python3.7/site-packages/asyncpg/connection.py", line 1411, in __execute
return await self._do_execute(query, executor, timeout)
File "/Users/son/Library/Caches/pypoetry/virtualenvs/project-rain-7LNSaARJ-py3.7/lib/python3.7/site-packages/asyncpg/connection.py", line 1423, in _do_execute
stmt = await self._get_statement(query, None)
File "/Users/son/Library/Caches/pypoetry/virtualenvs/project-rain-7LNSaARJ-py3.7/lib/python3.7/site-packages/asyncpg/connection.py", line 328, in _get_statement
statement = await self._protocol.prepare(stmt_name, query, timeout)
File "asyncpg/protocol/protocol.pyx", line 163, in prepare
asyncpg.exceptions.FeatureNotSupportedError: at or near "select": syntax error: unimplemented: this syntax
DETAIL: source SQL:
WITH RECURSIVE typeinfo_tree(
oid, ns, name, kind, basetype, has_bin_io, elemtype, elemdelim,
range_subtype, elem_has_bin_io, attrtypoids, attrnames, depth)
AS (
SELECT
ti.oid, ti.ns, ti.name, ti.kind, ti.basetype, ti.has_bin_io,
ti.elemtype, ti.elemdelim, ti.range_subtype, ti.elem_has_bin_io,
ti.attrtypoids, ti.attrnames, 0
FROM
(
SELECT
t.oid AS oid,
ns.nspname AS ns,
t.typname AS name,
t.typtype AS kind,
(CASE WHEN t.typtype = 'd' THEN
(WITH RECURSIVE typebases(oid, depth) AS (
SELECT
t2.typbasetype AS oid,
0 AS depth
FROM
pg_type t2
WHERE
t2.oid = t.oid

UNION ALL

SELECT
t2.typbasetype AS oid,
tb.depth + 1 AS depth
FROM
pg_type t2,
typebases tb
WHERE
tb.oid = t2.oid
AND t2.typbasetype != 0
) SELECT oid FROM typebases ORDER BY depth DESC LIMIT 1)
^
HINT: You have attempted to use a feature that is not yet implemented.
See: https://github.com/cockroachdb/cockroach/issues/21085
```

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the introspection query in asyncpg/introspection.py around lines 99–134 and the statement handling in asyncpg/connection.py around lines 328–338. Reproduce the INSERT with ARRAY columns against CockroachDB and trace the failing recursive type query; done means establishing compatible behavior or documenting the limitation with regression coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.