trinodb / trinodb/trino-python-client
Empty results on large queries when using fetchall()
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 439
- Forks
- 206
- PR merge metrics
- No merged PRs in 30d
Description
Hello there.
I've found that these two examples will not always return the same thing on large queries:
- fetchall
cur.execute(large_query)
rows = cur.fetchall()
print(rows)
- iterator
for r in cur.execute(large_query):
print(r)
The fetchall approach ends up returning an empty list every time I use it when running large queries, but the iterator approach prints the result successfully
Does anyone know why this may be happening?
I don't know how I should supply a working example to debug this problem, as my queries all contain domain logic of my company. Nevertheless, I can say that I'm connecting around 8 Postgres databases with 100k rows per table and equal schemas, and I union the tables from all databases together with UNION views, like this:
CREATE OR REPLACE VIEW hive.default."table_unioned_view" AS
SELECT 'database1' AS database, "other_column" FROM database1.public.table
UNION ALL
SELECT 'database2' AS database, "other_column" FROM database2.public.table
UNION ALL
SELECT 'database3' AS database, "other_column" FROM database3.public.table;
And the big query that I'm running uses these type of views
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by comparing the cursor.execute/fetchall path with cursor iteration using a minimal large-query example, since the report cannot provide a reproducible query. Check whether the behavior depends on the UNION views across the eight PostgreSQL databases; done means identifying and reproducing the cause of the differing results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgres, python
- Domain
- database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100