MagicStack / MagicStack/asyncpg
set_type_codec() doesn't work with copy_records_to_table()
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 468
- PR merge metrics
- No merged PRs in 30d
Description
- asyncpg version: 0.23.0
- PostgreSQL version: 9.5.21
- Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
the issue with a local PostgreSQL install?: No and this is on a local PSQL install. - Python version: 3.6.13
- Platform: Ubuntu 16.04.6 LTS
- Do you use pgbouncer?: No
- Did you install asyncpg with pip?: Yes
- If you built asyncpg locally, which version of Cython did you use?: n/a
- Can the issue be reproduced under both asyncio and
uvloop?: Not tried
The following:
await conn.set_type_codec(
"numeric",
encoder=str,
decoder=float,
schema="pg_catalog",
)
Doesn't work when used with copy_records_to_table()
...
File "asyncpg/protocol/protocol.pyx", line 504, in copy_in
File "asyncpg/protocol/protocol.pyx", line 437, in asyncpg.protocol.protocol.BaseProtocol.copy_in
asyncpg.exceptions._base.InternalClientError: no binary format encoder for type numeric (OID 1700)
This is despite solving issue https://github.com/MagicStack/asyncpg/issues/157 which added a binary format codec for numeric types - https://github.com/MagicStack/asyncpg/commit/0f8483517ebde66f3c7cc858fa6ad227c6e3fd64
Can we get the ability to provide a custom decoder without having to also provide an encoder then internally just fall back to the default encoder? I believe it should solve the issue. E.g.
await conn.set_type_codec(
"numeric",
decoder=float,
schema="pg_catalog",
)
or
await conn.set_type_codec(
"numeric",
encoder=None,
decoder=float,
schema="pg_catalog",
)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with set_type_codec() and copy_records_to_table() using the issue's numeric example. Start in asyncpg/protocol/protocol.pyx at copy_in and BaseProtocol.copy_in, then trace how the custom codec is selected. Done means a custom decoder can be used while copy_records_to_table() still obtains an encoder for numeric.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, python
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100