ClickHouse / ClickHouse/clickhouse-rs

Add `Query::fetch_owned`

Open
#401 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
559
Forks
172
Avg merge
1d 19h
Merged PRs (30d)
3

Description

We are using `Query::fetch` as we have data we want to select that is too big to fit into memory. Unfortunately, it seems the implementation buffers the whole response rather than reusing the memory. I understand the lifetime-related rationale for this as outlined in #24.

In #247 the `RowOwned` type was added to help in these scenarios. A function similar to `Query::fetch` but that is constrained to `RowOwned` such that it can reuse the memory would solve our scenario and allow querying massive or infinite data sets.

_(We cannot split the query up at the moment as we use FINAL and rely on the atomic nature of the single query.)_

Thanks

Contributor guide

Open the contributing guide

Research direction

Start by reading the existing Query::fetch implementation and the RowOwned type added in #247, then review the lifetime rationale in #24. Trace how rows are buffered and identify the relevant query tests or entry points. Done means a fetch_owned variant can stream massive or infinite results while reusing memory and preserves the single-query behavior needed with FINAL.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.