ClickHouse / ClickHouse/clickhouse-rs
Add `Query::fetch_owned`
- Dominant language
- Rust
- Stars
- 559
- Forks
- 172
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 3
Description
We are using `Query::fetch` as we have data we want to select that is too big to fit into memory. Unfortunately, it seems the implementation buffers the whole response rather than reusing the memory. I understand the lifetime-related rationale for this as outlined in #24.
In #247 the `RowOwned` type was added to help in these scenarios. A function similar to `Query::fetch` but that is constrained to `RowOwned` such that it can reuse the memory would solve our scenario and allow querying massive or infinite data sets.
_(We cannot split the query up at the moment as we use FINAL and rely on the atomic nature of the single query.)_
Thanks
Contributor guide
Research direction
Start by reading the existing Query::fetch implementation and the RowOwned type added in #247, then review the lifetime rationale in #24. Trace how rows are buffered and identify the relevant query tests or entry points. Done means a fetch_owned variant can stream massive or infinite results while reusing memory and preserves the single-query behavior needed with FINAL.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100