MagicStack / MagicStack/asyncpg
decimal.InvalidOperation exception is thrown with AWS Redshift.
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 468
- PR merge metrics
- No merged PRs in 30d
Description
* **asyncpg version**: 0.23.0
* **PostgreSQL version**: PostgreSQL 8.0.2 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.4.2 20041017 (Red Hat 3.4.2-6.fc3), Redshift 1.0.27314
* **Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
the issue with a local PostgreSQL install?**: AWS Redshift.
* **Python version**: 3.6.13
* **Platform**: Linux 5.0.0-23-generic #24~18.04.1-Ubuntu SMP Mon Jul 29 16:12:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
* **Do you use pgbouncer?**: no
* **Did you install asyncpg with pip?**: yes
* **If you built asyncpg locally, which version of Cython did you use?**: no
* **Can the issue be reproduced under both asyncio and
[uvloop](https://github.com/magicstack/uvloop)?**: no
I am trying to use asyncpg with AWS Redshift. I got an decimal.InvalidOperation exception when I try to run a simple query as follows. Is there any way to make asyncpg work with AWS Redshift? psycopg2 is working with AWS Redshift without major issue so I expect asyncpg could work with AWS Redshift also. Thank you in advance.
```
import asyncio
import asyncpg
async def run():
conn = await asyncpg.connect(
user='user', password='password', database='database', host='host')
values = await conn.fetch('SELECT 1.0 / 0.5')
await conn.close()
loop = asyncio.get_event_loop()
loop.run_until_complete(run())
```
```
$ python asyncpg_redshift.py
Traceback (most recent call last):
File "asyncpg_redshift.py", line 11, in
loop.run_until_complete(run())
File "/home/taeho/.pyenv/versions/3.6.13/lib/python3.6/asyncio/base_events.py", line 488, in run_until_complete
return future.result()
File "asyncpg_redshift.py", line 7, in run
values = await conn.fetch('SELECT 1.0 / 0.5')
File "/home/taeho/.pyenv/versions/3.6.13/envs/asyncpg/lib/python3.6/site-packages/asyncpg/connection.py", line 588, in fetch
record_class=record_class,
File "/home/taeho/.pyenv/versions/3.6.13/envs/asyncpg/lib/python3.6/site-packages/asyncpg/connection.py", line 1632, in _execute
ignore_custom_codec=ignore_custom_codec,
File "/home/taeho/.pyenv/versions/3.6.13/envs/asyncpg/lib/python3.6/site-packages/asyncpg/connection.py", line 1655, in __execute
ignore_custom_codec=ignore_custom_codec,
File "/home/taeho/.pyenv/versions/3.6.13/envs/asyncpg/lib/python3.6/site-packages/asyncpg/connection.py", line 1697, in _do_execute
result = await executor(stmt, None)
File "asyncpg/protocol/protocol.pyx", line 201, in bind_execute
File "asyncpg/protocol/coreproto.pyx", line 70, in asyncpg.protocol.protocol.CoreProtocol._read_server_messages
File "asyncpg/protocol/coreproto.pyx", line 212, in asyncpg.protocol.protocol.CoreProtocol._process__bind_execute
File "asyncpg/protocol/coreproto.pyx", line 497, in asyncpg.protocol.protocol.CoreProtocol._parse_data_msgs
File "asyncpg/protocol/protocol.pyx", line 781, in asyncpg.protocol.protocol.BaseProtocol._decode_row
File "asyncpg/protocol/prepared_stmt.pyx", line 295, in asyncpg.protocol.protocol.PreparedStatementState._decode_row
File "asyncpg/protocol/codecs/base.pyx", line 289, in asyncpg.protocol.protocol.Codec.decode
File "asyncpg/protocol/codecs/base.pyx", line 196, in asyncpg.protocol.protocol.Codec.decode_scalar
File "asyncpg/pgproto/./codecs/numeric.pyx", line 287, in asyncpg.pgproto.pgproto.numeric_decode_binary
File "asyncpg/pgproto/./codecs/numeric.pyx", line 279, in asyncpg.pgproto.pgproto.numeric_decode_binary_ex
decimal.InvalidOperation: []
$
```
I think the strange result of the following query could be also related.
```
import asyncio
import asyncpg
async def run():
conn = await asyncpg.connect(
user='user', password='password', database='database', host='host')
values = await conn.fetch('SELECT CAST(1.0 AS DECIMAL(10, 1))')
print(values)
await conn.close()
loop = asyncio.get_event_loop()
loop.run_until_complete(run())
```
```
$ python asyncpg_redshift2.py
[]
$
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by running the two provided Python queries against AWS Redshift and compare their results with the traceback. Inspect asyncpg/pgproto/./codecs/numeric.pyx, where the failure occurs during numeric decoding, and determine how Redshift’s returned numeric data differs from PostgreSQL’s expected representation. Done means the incompatibility is reproduced and a supported behavior or narrowly scoped fix is established.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, postgresql, python
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100