tortoise / tortoise/tortoise-orm
select_related not preserved in ValuesQuery
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5.6k
- Forks
- 516
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 9
Description
Hello!
I noticed that the optimization done in ValuesQuery is a bit too agressive when it comes to related selects.
Take the following example:
from tortoise.models import Model
from tortoise import Tortoise, fields
class Author(Model):
age = fields.IntField()
class Book(Model):
author = fields.ForeignKeyField("models.Author")
async def main():
await Tortoise.init(db_url='sqlite://:memory:', modules={"models": ["example"]})
query = (
Book.all()
.select_related("author")
.annotate(custom=RawSQL("book__author.age * 2 :: BIGINT"))
.order_by("custom")
)
print(query.sql())
# SELECT "book"."id","book"."author_id",book__author.age * 2 :: BIGINT "custom","book__author"."id" "book__author.id","book__author"."age" "book__author.age" FROM "book" LEFT OUTER JOIN "author" "book__author" ON "book__author"."id"="book"."author_id" ORDER BY "custom" ASC
print(query.values("id").sql())
# SELECT "id" "id" FROM "book" ORDER BY book__author.age * 2 :: BIGINT ASC
As you can see in the second query, the JOIN is removed which results in an operational error, as book__author doesn't exist anymore. It seems Tortoise isn't aware that I need the join, despite having requested it explicitely via select_related.
I can imagine it's hard for the library to understand which related fields are needed when RawSQL is being used (I'm forced to use it for the cast, there's no way to my knowledge to cast types ORM-style), is there a way for me to force Tortoise to make that join?
Edit: I'd rather avoid having more than one column in the .values, I'm actually using .values_list("id", flat=True), which I can't do with two values.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the ValuesQuery optimization involved in values() and values_list(), then reproduce the provided select_related, RawSQL, and order_by example against SQLite. Trace why the related join is removed and verify that the resulting SQL preserves the join needed by the RawSQL ordering while still returning only the requested value column.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sqlite
- Domain
- database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100