NVIDIA / NVIDIA/cudf

[BUG] [Java] Memory leak due to table writers never being destroyed after done writing

Open
#17,720 0 comments 0 reactions 0 assignees View on GitHub
bug Java Spark
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

For all kinds of table writers (Java code: `Table.java`, JNI code: `TableJni.cpp`), the native C++ instances are never destroyed after calling `close()` in the corresponding Java classes. For example:
```
@Override
public void close() throws CudfException {
if (writerHandle != 0) {
writeParquetEnd(writerHandle);
}
writerHandle = 0;
```

And here is the C++ function `writeParquetEnd`:
```
try {
cudf::jni::auto_set_device(env);
state->writer->close();
}
```

Note that `state` is an unmanaged pointer that corresponds to `writerHandle` in Java and stores `writer` as a `unique_ptr`. So the `writer` instance is destroyed automatically when `state` is destroyed, however, `state` is never destroyed.

When `writeParquetEnd` is called, the C++ code only calls the member `close()` function of the writer, never calls `delete` on the `state` instance. After this, the Java code simply leaves that native instance dangled.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.