cloudflare / cloudflare/sqlalchemy-clickhouse
Is there a way to stream large result set
Open
- Dominant language
- Python
- Stars
- 325
- Forks
- 101
- PR merge metrics
- No merged PRs in 30d
Description
Currently the `Cursor.execute` will retrieve all the result rows into memory, this could cause memory and performance issues if the result rows is very large (e.g. `SELECT * FROM ontime`).
Is there a better solution for this situation?
Contributor guide
Research direction
Start by inspecting the Cursor.execute entry point mentioned in the issue and trace how result rows are retrieved and stored. Compare the available SQLAlchemy result-streaming patterns and existing tests, if any. Done means large queries can be consumed without loading every row into memory, with behavior covered by tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sqlalchemy
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100