gluten-velox-bundle.jar NOT working with provided --conf spark.plugins=org.apache.gluten.GlutenPlugin \ with spark-shell tried io.glutenproject.GlutenPlugin instead (exists in JAR)
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Problem description
```
$SPARK_HOME/bin/spark-shell \ 127 ✘
--master spark://localhost:7077 \
--deploy-mode client \
--conf spark.plugins=io.glutenproject.GlutenPlugin \
--conf spark.memory.offHeap.enabled=true \
--conf spark.memory.offHeap.size=20g \
--conf spark.shuffle.manager=org.apache.spark.shuffle.sort.ColumnarShuffleManager \
--jars /home/agentperry/Documents/Zettabolt/spark-3.2.4-bin-hadoop3.2/jars/gluten-velox-bundle-spark3.2_2.12-1.1.1.jar
2024-04-08 16:40:55,238 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
OpenJDK 64-Bit Server VM warning: You have loaded library /tmp/gluten-b515788b-b16c-4cd4-98e9-43cbf50fd401/jni/0d51f686-650e-46a1-a8a4-1c767df4c01d/gluten-7964092948296879699/libvelox.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'.
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGILL (0x4) at pc=0x000077d021717353, pid=53799, tid=0x000077d056fff640
#
# JRE version: OpenJDK Runtime Environment (8.0_402-b06) (build 1.8.0_402-8u402-ga-2ubuntu1~22.04-b06)
# Java VM: OpenJDK 64-Bit Server VM (25.402-b06 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C [libgluten.so+0x317353] gluten::Runtime::registerFactory(std::string const&, std::function, std::equal_to, std::allocator > > const&)>)+0x23
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/agentperry/hs_err_pid53799.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
```
### System information
#########################
# Monday 08 April, 2024 #
#########################
agentperry@warmachine86-2 (20Y700B5IG)
Ubuntu 22.04.4 LTS (GNU/Linux 6.5.0-26-generic)
Arch: x86_64
Packages: 2221 (dpkg)
Shell: Z Shell
Public IPv4: (AS17488 - Hathway IP Over Cable Internet - IN)
Local IP: 192.168.1.8 172.17.0.1
### CMake log
```bash
$SPARK_HOME/bin/spark-shell \ 127 ✘
--master spark://localhost:7077 \
--deploy-mode client \
--conf spark.plugins=io.glutenproject.GlutenPlugin \
--conf spark.memory.offHeap.enabled=true \
--conf spark.memory.offHeap.size=20g \
--conf spark.shuffle.manager=org.apache.spark.shuffle.sort.ColumnarShuffleManager \
--jars /home/agentperry/Documents/Zettabolt/spark-3.2.4-bin-hadoop3.2/jars/gluten-velox-bundle-spark3.2_2.12-1.1.1.jar
2024-04-08 16:40:55,238 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
OpenJDK 64-Bit Server VM warning: You have loaded library /tmp/gluten-b515788b-b16c-4cd4-98e9-43cbf50fd401/jni/0d51f686-650e-46a1-a8a4-1c767df4c01d/gluten-7964092948296879699/libvelox.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'.
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGILL (0x4) at pc=0x000077d021717353, pid=53799, tid=0x000077d056fff640
#
# JRE version: OpenJDK Runtime Environment (8.0_402-b06) (build 1.8.0_402-8u402-ga-2ubuntu1~22.04-b06)
# Java VM: OpenJDK 64-Bit Server VM (25.402-b06 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C [libgluten.so+0x317353] gluten::Runtime::registerFactory(std::string const&, std::function, std::equal_to, std::allocator > > const&)>)+0x23
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/agentperry/hs_err_pid53799.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
/home/agentperry/Documents/Zettabolt/spark-3.2.4-bin-hadoop3.2//bin/spark-shell: line 47: 53799 Aborted (core dumped) "${SPARK_HOME}"/bin/spark-submit --class org.apache.spark.repl.Main --name "Spark shell" "$@"
```
Contributor guide
Research direction
Reproduce the failure with the supplied spark-shell command and gluten-velox-bundle-spark3.2_2.12-1.1.1.jar on the stated Ubuntu and Java 8 environment. Start with hs_err_pid53799.log and the reported libgluten.so SIGILL in gluten::Runtime::registerFactory; done means spark-shell starts without the native crash.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, shell
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100