tortoise / tortoise/tortoise-orm
Erroneous update query
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5.6k
- Forks
- 516
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 9
Description
Describe the bug
An erroneous update query is created as per below. This leads to an error instead of an update...
To Reproduce
await CPEVlan.filter(vlan=entry['vlan'], cpe__mac=entry['mac'].replace(':', '').upper())\
.update(lease=entry['lease'], ip=entry['ip'], seen_in_ipv6_snooping=seen_time, last_seen=seen_time)
Creates the following query:
UPDATE "network_cpevlan" LEFT OUTER JOIN "network_cpe" "network_cpevlan__cpe" ON "network_cpevlan__cpe"."id"="network_cpevlan"."cpe_id" SET "lease"=<snip>,"ip"='<snip>',"seen_in_ipv6_snooping"=true,"last_seen"='2020-06-16T13:50:26.436003+00:00' WHERE "network_cpevlan"."vlan"=<snip> AND "network_cpevlan__cpe"."mac"='<snip>'
Which fails with the following stacktrace:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/tortoise/__init__.py", line 636, in run_async
loop.run_until_complete(coro)
File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/opt/project/locator/network/parsing/snooping.py", line 38, in timetest
await parse_ipv4_snooping(ipv4_snooping_data)
File "/opt/project/locator/network/parsing/snooping.py", line 24, in parse_ipv4_snooping2
await CPEVlan.filter(vlan=entry['vlan'], cpe__mac=entry['mac'].replace(':', '').upper())\
File "/usr/local/lib/python3.8/site-packages/tortoise/queryset.py", line 793, in _execute
return (await self._db.execute_query(str(self.query)))[0]
File "/usr/local/lib/python3.8/site-packages/tortoise/backends/asyncpg/client.py", line 38, in translate_exceptions_
raise OperationalError(exc)
tortoise.exceptions.OperationalError: syntax error at or near "LEFT"
On the following model:
class CPEVlan(models.Model):
cpe: fields.ForeignKeyRelation[CPE] = fields.ForeignKeyField(
'network.CPE', on_delete=fields.CASCADE, related_name='vlans'
)
vlan = fields.IntField()
seen_in_snooping = fields.BooleanField(default=False)
seen_in_ipv6_snooping = fields.BooleanField(default=False)
seen_in_mac_table = fields.BooleanField(default=False)
last_seen = fields.DatetimeField()
lease = fields.FloatField(null=True)
ipv6_lease = fields.FloatField(null=True)
ip = fields.CharField(max_length=15, null=True)
ipv6_ip = fields.CharField(max_length=45, null=True)
ipv6_range = fields.CharField(max_length=45, null=True)
class Meta:
table = 'network_cpevlan'
unique_together =[
['cpe', 'vlan']
]
Expected behavior
Expected the update to succeed update-ing, and not to fail.
Additional context
Running against postgresql.
Tortoise-orm: 0.16.13
Python: 3.8.3
Rest of the packages are all up-to-date:
PACKAGE VERSION NEWEST_VERSION
PyPika 0.37.7 0.37.7
aiohttp 3.6.2 3.6.2
aiohttp-cors 0.7.0
aiosqlite 0.13.0 0.13.0
async-timeout 3.0.1 3.0.1
asyncpg 0.20.1 0.20.1
attrs 19.3.0 19.3.0
certifi 2020.4.5.2 2020.4.5.2
chardet 3.0.4 3.0.4
click 7.1.2 7.1.2
colorclass 2.2.0 2.2.0
colorlog 4.1.0 4.1.0
croniter 0.3.33 0.3.33
faust 1.10.4 1.10.4
idna 2.9 2.9
iso8601 0.1.12 0.1.12
kafka-python 1.4.7 2.0.1
mode 4.3.2 4.3.2
multidict 4.7.6 4.7.6
mypy-extensions 0.4.3 0.4.3
natsort 7.0.1 7.0.1
opentracing 1.3.0 2.3.0
pip 20.1.1 20.1.1
python-dateutil 2.8.1 2.8.1
robinhood-aiokafka 1.1.6 1.1.6
sentry-sdk 0.14.4 0.14.4
setuptools 47.1.1 47.3.0
six 1.15.0 1.15.0
terminaltables 3.1.0 3.1.0
tortoise-orm 0.16.13 0.16.13
typing-extensions 3.7.4.2 3.7.4.2
urllib3 1.25.9 1.25.9
venusian 1.2.0 3.0.0
wheel 0.34.2 0.34.2
yarl 1.4.2 1.4.2
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the update execution path in tortoise/queryset.py and the PostgreSQL backend shown in backends/asyncpg/client.py, reproducing the CPEVlan update with a related cpe filter. Done means the generated PostgreSQL UPDATE no longer places an invalid LEFT OUTER JOIN before SET and the update succeeds.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, python
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100