tortoise / tortoise/tortoise-orm

Group by for many to many table problem

Open
#1,533 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
5.6k
Forks
516
Avg merge
2d 21h
Merged PRs (30d)
9

Description

Hi,
I am encountering an issue with how Tortoise ORM handles grouping in queries involving many-to-many tables. Specifically, when I attempt to group by a single field in a many-to-many intermediary table, the SQL query produced by Tortoise ORM includes additional fields in the GROUP BY clause, leading to incorrect aggregation results.

this is the model class

class Recipe2Ingredient(Model):
    class Meta:
        app = "recipes"
        table = "recipes_recipe2ingredient"

    id = fields.UUIDField(pk=True)
    recipe = fields.ForeignKeyField(
        'recipes.Recipe',
        related_name='ingredients'
    )
    ingredient = fields.ForeignKeyField(
        'recipes.Ingredient',
        related_name='recipes'
    )

this is the query

query = models.Recipe2Ingredient.annotate(
                count=Count("ingredient_id")
            ).group_by("ingredient_id").order_by("-count").limit(5).prefetch_related("ingredient")
          

I expected this query to produce raw sql query:

SELECT "ingredient_id", COUNT("ingredient_id") AS "count"
FROM "recipes_recipe2ingredient"
GROUP BY "ingredient_id"
ORDER BY "count" DESC
LIMIT 5

but I got

'SELECT "recipe_id","id","ingredient_id",COUNT("ingredient_id") "count" FROM "recipes_recipe2ingredient" 
GROUP BY "recipe_id","id","ingredient_id" 
ORDER BY COUNT("ingredient_id") DESC
 LIMIT 5'

My main problem is with the grouping part because instead of getting GROUP BY "ingredient_id" I got GROUP BY "recipe_id","id","ingredient_id" and this is totally incorrect.

Is there something I was doing wrong or is this is a tortoise bug? If so, can you please fix this bug.

Thanks

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start from the Recipe2Ingredient model and the annotate().group_by("ingredient_id") query shown in the report, then trace how Tortoise builds SELECT and GROUP BY clauses for relational fields. Reproduce the generated SQL and verify that grouping only by ingredient_id produces the intended aggregation and ordering.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
database
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.