group_by produces invalid query on invalid input

Describe the bug
If I make a complex query with conditions that will not work, then instead of erroring Tortoise will change parts of the query in order to produce a valid but incorrect query.

To Reproduce

With the given table definition:

class MangaStat(Model):
    rowid = IntField(pk=True)
    id = UUIDField(null=False, index=True)
    time = DatetimeField(null=False)

    rating = DecimalField(6, 4, default=0)
    followers = IntField(null=False, default=0)

I tried to execute the code:

query = await MangaStat.all().annotate(cnt=Count("id")).filter(cnt__gt=20000).group_by("time").order_by("-time").limit(2)

The resulting SQL was:

SELECT "time", "rowid", "id", "rating", "followers", COUNT(*) "cnt"
FROM "mangastat"
GROUP BY "rowid"
HAVING COUNT(*) > 20000
ORDER BY "time" DESC

Expected behavior
The query manager should have thrown an exception when trying to run an invalid query instead of trying to fix it.

Additional context
If I manually changed rowid to time in the GROUP BY cause I got an error as expected.

Use the provided MangaStat model and annotated group_by query as the reproduction case, then inspect the query manager's handling of invalid GROUP BY input. Done means the query raises an exception instead of changing the GROUP BY to rowid and producing an incorrect query.