MagicStack / MagicStack/asyncpg
no binary format encoder for type jsonb
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 468
- PR merge metrics
- No merged PRs in 30d
Description
* **asyncpg version**: 0.29.0
* **PostgreSQL version**: 16
* **Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
the issue with a local PostgreSQL install?**:
* **Python version**: 3.11
* **Platform**: 5.10.0-21-arm64 #1 SMP Debian 5.10.162-1 (2023-01-21) aarch64 GNU/Linux
* **Do you use pgbouncer?**:no
* **Did you install asyncpg with pip?**: yes
* **If you built asyncpg locally, which version of Cython did you use?**:
* **Can the issue be reproduced under both asyncio and
[uvloop](https://github.com/magicstack/uvloop)?**: yes
,asyncpg.exceptions._base.InternalClientError: no binary format encoder for type jsonb (OID 3802)
```
async def insert_binary(self, table, rows):
if rows:
rows = common_function.convert_data_type(rows, decimal.Decimal, float, )
columms = rows[0].keys()
data = []
for row_index, rows_once in enumerate(rows):
data_once = [rows_once[columns_once] for columns_once in columms]
data.append(tuple(data_once))
conn = await self.connect()
await conn.copy_records_to_table(table,records=data, columns=rows[0].keys())
await conn.fetch('SELECT 1')
await conn.close()
```
I've tried various methods, but I couldn't manage to insert binary data. I've tried using decoders and other approaches. It seems like it needs a binary decoder, not a text one, but I haven't figured out how to represent it in binary. I'd appreciate any help.
```
await conn.set_type_codec(
"jsonb",
encoder=lambda data: b"\x01" + common_function.decoder.dumps(data, False),
decoder=lambda data: common_function.decoder.loads(data[1:]),
schema="pg_catalog",
format="binary"
)
await conn.set_type_codec(
'numeric'
, encoder = lambda f: str(f).encode('utf8')
, decoder = lambda b: b.decode('utf8')
, schema='pg_catalog'
, format='binary'
)
```
"This also doesn't work."
"I replaced 'numeric' with 'float' (database field types), and the problem 'no binary format encoder for type numeric' disappeared. However, I couldn't figure out which format decoder is needed for JSON."
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with copy_records_to_table and set_type_codec, reproducing the reported failure for PostgreSQL jsonb (OID 3802) with binary format. Trace the existing JSON and numeric codec handling, then verify that binary-format jsonb records are accepted without the reported encoder error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, python
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100