apache / apache/datafusion-comet
Possible memory leak in error cases in JVM-Rust (and vice versa) transition
Open
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 373
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 190
Description
Looking at the code, it seems like there might be a memory leak when there are errors, probably there are more
https://github.com/apache/datafusion-comet/blob/d3ea9fdd505fdda71c8de4b3d2021a4ba1f99ae1/native/core/src/execution/operators/scan.rs#L205
I think there should be some more robust way to handle arrow over FFI that avoid memory leaks (maybe move all JNI arrow stuff behind a guard that on drop will release the memory or calling `to_owned` will take the memory)
Contributor guide
Assessment
This issue has not been assessed yet.