apache / apache/arrow-java

JVM failed when use gandiva udf with dynamic libraries

Open
#326 0 comments 0 reactions 0 assignees View on GitHub
Type: bug
Dominant language
Java
Stars
94
Forks
152
Avg merge
3d 16h
Merged PRs (30d)
11

Description

Hi there,

Recently I'm trying to add some UDF with dynamic link libaries. It is fine compiling and running test in cpp, but when I call the udf from java, JVM failed with errors.

Steps to reproduce the issue
1 Prepare dynamic library 'libmytest.so'
```java

// code placeholder
#ifndef MYTEST_H
#define MYTEST_H
#ifdef __cplusplus
extern "C"{
#endif
float testSim();
#ifdef __cplusplus
}
#endif
#endif
```
2 Add simple code for the udf in file 'string_ops.cc'

 
```java

// code placeholder
FORCE_INLINEFORCE_INLINE
gdv_float32 test_sim_binary_binary(gdv_int64 context, const char* left, gdv_int32 left_len, const char* right,                 gdv_int32 right_len) {
float sim = testSim();
return sim;
}
```
 

3 Add function details in the function registry file 'function_registry_string.cc'
```java

// code placeholder
NativeFunction("test_sim", {}, DataTypeVector{binary(),binary()},float32(),
kResultNullIfNull, "sim_binary_binary", NativeFunction::kNeedsContext | NativeFunction::kCanReturnErrors)
```
4 Create test functions

5 Add link to the CMakeLists.txt

5 compile and test

6 write a java demo to call the udf

[hs_err_pid28288.log](hs_err_pid28288.log)

**Environment**: OS:Centos7.4
llvm:7.0.1
jdk:1.8.0_162
arrow:1.0.0
**Reporter**: [Leo89](https://issues.apache.org/jira/browse/ARROW-9624)
#### Original Issue Attachments:
- [hs_err_pid28288.log](https://issues.apache.org/jira/secure/attachment/13008955/hs_err_pid28288.log)

**Note**: *This issue was originally created as [ARROW-9624](https://issues.apache.org/jira/browse/ARROW-9624). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start by reviewing hs_err_pid28288.log and reproducing the Java call to the Gandiva UDF with libmytest.so. Trace the native UDF changes in string_ops.cc, function_registry_string.cc, and CMakeLists.txt, comparing the C++ and Java paths. Done means the Java demo no longer causes the JVM failure while the dynamic-library UDF remains callable.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, java
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.