tortoise / tortoise/tortoise-orm

bulk_update fails when a jsonb field contains an array with a single quote entry under postgres

Open
#1,067 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
5.6k
Forks
516
Avg merge
2d 21h
Merged PRs (30d)
9

Description

Describe the bug
Updating a jsonb field via bulk_update with an array containing string values with single quotes, the resulting SQL is incorrect due to lack of escaping which leads to the update failure.

To Reproduce

I have not yet verified if this is relevant, but in my application code two conditions are present:

  1. I am updating an entry previously fetched form a database
  2. Field is updated to a top level array
class MemeModel(Model):
    tags = fields.JSONField()

meme = MemeModel.all().first()
meme.tags = ["it's friday my dudes"]
MemeModel.bulk_update([meme], fields=['tags'])
Traceback (most recent call last):
  File "tortoise\backends\asyncpg\client.py", line 36, in translate_exceptions_
    return await func(self, *args)
  File "tortoise\backends\asyncpg\client.py", line 178, in execute_query
    res = await connection.execute(*params)
  File "asyncpg\connection.py", line 318, in execute
    return await self._protocol.query(query, timeout)
  File "asyncpg\protocol\protocol.pyx", line 338, in query
asyncpg.exceptions.PostgresSyntaxError: syntax error at or near "s"
... "tags"=CASE WHEN "id"=123 THEN '["it's friday my dudes"]'::jsonb END...

My understanding is that while json.dumps escapes double quotes, the single quote is still present un-escaped in the final string and conflicts with the ::jsonb cast which also uses single quotes. I suspect the issue may be mitigated either by using $$ for quoting or with an extra pass to escape the single quote within the result of json.dumps

Expected behavior
bulk_update succeeds

Additional context
I've verified that https://github.com/tortoise/tortoise-orm/pull/1023 (and the follow up commit https://github.com/tortoise/tortoise-orm/commit/1c8d30466b2a82914ccc731f6ac0d5755b682f4f + pypika git branch do not fix the issue).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the PostgreSQL execution path shown in tortoise/backends/asyncpg/client.py and trace how bulk_update constructs the quoted jsonb value. Reproduce the provided example with a single quote in the array, then add regression coverage; done means bulk_update succeeds and stores the expected JSONB value.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.