apache / apache/gluten

[VL] Pass additional configuration options programmatically from Java to C++

Open
#9,785 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

Enhance this API https://github.com/apache/incubator-gluten/blob/2feb5f936ccb81b11b053f27cbfbf653e1a3c742/gluten-arrow/src/main/java/org/apache/gluten/vectorized/NativePlanEvaluator.java#L46-L47 by adding a parameter that allows additional configuration options being passed to the native Velox runtime.

By doing this we could avoid adding JNI parameters every time we need some customization ([example](https://github.com/apache/incubator-gluten/pull/9634/files#diff-4f3b38d38be472d3634702bf87c6ac1a990997eafe4207f67ce48f7b5f550bb1R401)) for native execution.

For example, JNI signature

```cpp
Java_org_apache_gluten_vectorized_PlanEvaluatorJniWrapper_nativeCreateKernelWithIterator( // NOLINT
JNIEnv* env,
jobject wrapper,
jbyteArray planArr,
jobjectArray splitInfosArr,
jobjectArray iterArr,
jint stageId,
jint partitionId,
jlong taskId,
jboolean enableDumping,
jstring spillDir,
jboolean enableCudf)
```

may be simplified to

```cpp
Java_org_apache_gluten_vectorized_PlanEvaluatorJniWrapper_nativeCreateKernelWithIterator( // NOLINT
JNIEnv* env,
jobject wrapper,
jbyteArray planArr,
jobjectArray splitInfosArr,
jobjectArray iterArr)
```

once the feature is implemented and used.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.