tortoise / tortoise/tortoise-orm
bulk_update fails when a jsonb field contains an array with a single quote entry under postgres
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5.6k
- Forks
- 516
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 9
Description
Describe the bug
Updating a jsonb field via bulk_update with an array containing string values with single quotes, the resulting SQL is incorrect due to lack of escaping which leads to the update failure.
To Reproduce
I have not yet verified if this is relevant, but in my application code two conditions are present:
- I am updating an entry previously fetched form a database
- Field is updated to a top level array
class MemeModel(Model):
tags = fields.JSONField()
meme = MemeModel.all().first()
meme.tags = ["it's friday my dudes"]
MemeModel.bulk_update([meme], fields=['tags'])
Traceback (most recent call last):
File "tortoise\backends\asyncpg\client.py", line 36, in translate_exceptions_
return await func(self, *args)
File "tortoise\backends\asyncpg\client.py", line 178, in execute_query
res = await connection.execute(*params)
File "asyncpg\connection.py", line 318, in execute
return await self._protocol.query(query, timeout)
File "asyncpg\protocol\protocol.pyx", line 338, in query
asyncpg.exceptions.PostgresSyntaxError: syntax error at or near "s"
... "tags"=CASE WHEN "id"=123 THEN '["it's friday my dudes"]'::jsonb END...
My understanding is that while json.dumps escapes double quotes, the single quote is still present un-escaped in the final string and conflicts with the ::jsonb cast which also uses single quotes. I suspect the issue may be mitigated either by using $$ for quoting or with an extra pass to escape the single quote within the result of json.dumps
Expected behavior
bulk_update succeeds
Additional context
I've verified that https://github.com/tortoise/tortoise-orm/pull/1023 (and the follow up commit https://github.com/tortoise/tortoise-orm/commit/1c8d30466b2a82914ccc731f6ac0d5755b682f4f + pypika git branch do not fix the issue).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the PostgreSQL execution path shown in tortoise/backends/asyncpg/client.py and trace how bulk_update constructs the quoted jsonb value. Reproduce the provided example with a single quote in the array, then add regression coverage; done means bulk_update succeeds and stores the expected JSONB value.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, python
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100