apache / apache/arrow-java

[JAVA] Leak in JdbcToArrowUtils

Open
#408 1 comment 0 reactions 0 assignees View on GitHub
Type: bug
Dominant language
Java
Stars
94
Forks
152
Avg merge
3d 16h
Merged PRs (30d)
11

Description

`JdbcToArrowUtils::updateVector(VarCharVector, String, boolean, int)` does not release the memory that it allocates for the `NullableVarCharHolder`. This can be verified by changing the first lines of `JdbcToArrowTest::testJdbcToArrowValues()` to the following:
```java

RootAllocator allocator = new RootAllocator(Integer.MAX_VALUE);
VectorSchemaRoot root = JdbcToArrow.sqlToArrow(conn, table.getQuery(), allocator, Calendar.getInstance());
testDataSets(root);
root.close();
assertEquals(allocator.getAllocatedMemory(), 0);
```
 

The leak can be fixed by closing the buffer in `updateVector`. However, I would propose to not use the `NullableVarCharHolder` in the first place because it creates an unnecessary copy of the string. Instead, I would use `BaseVariableWidthVector::setSafe(int index, byte[] value)` and `BaseVariableWidthVector::setNull(int index)` directly.

In addition, I would propose to check for leaks in JdbcToArrowTest.

I am willing to create a patch, so please let me know what approach is preferred.

**Reporter**: [Johannes Luong](https://issues.apache.org/jira/browse/ARROW-5536) / @jmaschad

**Note**: *This issue was originally created as [ARROW-5536](https://issues.apache.org/jira/browse/ARROW-5536). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start in JdbcToArrowUtils::updateVector(VarCharVector, String, boolean, int) and inspect how NullableVarCharHolder allocates its buffer. Review JdbcToArrowTest::testJdbcToArrowValues() and add the allocator-memory assertion shown in the issue; the work is done when the test passes with zero allocated memory after closing the root.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.