apache / apache/arrow-adbc

java jdbc option FetchSize improvement

Open
#2,156 0 comments 0 reactions 0 assignees View on GitHub
Type: enhancement
Dominant language
C#
Stars
627
Forks
217
Avg merge
17h
Merged PRs (30d)
57

Description

### What feature or improvement would you like to see?

according to https://arrow.apache.org/adbc/current/java/quickstart.html

```
final Map parameters = new HashMap<>();
parameters.put(AdbcDriver.PARAM_URL, "jdbc:postgresql://localhost:5432/postgres");
try (
BufferAllocator allocator = new RootAllocator();
AdbcDatabase db = new JdbcDriver(allocator).open(parameters);
AdbcConnection adbcConnection = db.connect();
AdbcStatement stmt = adbcConnection.createStatement()

) {
stmt.setSqlQuery("select * from foo");
AdbcStatement.QueryResult queryResult = stmt.executeQuery();
while (queryResult.getReader().loadNextBatch()) {
// process batch
}
} catch (AdbcException e) {
// throw
}

```

stmt.executeQuery() will cause oom


in jdbc, statement has a method `statement.setFetchSize(batchSize);` to fetch a part of data
can give a option to define this and `queryResult.getReader().loadNextBatch()` load this size data

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.