[VL] libgluten.so crash while building gluten velox
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Problem description
**Backend**
Velox
**Bug description**
I built velox gluten according to the requirements of the official website, but encountered an error of 'C [libgluten. so+0x317353]' during startup
**Reproduction**
**Create the docker container**
```
docker pull ubuntu:22.04
docker run -itd --name ubuntu2204 ubuntu:22.04 /bin/bash
docker attach ubuntu2204
apt-get update
apt install software-properties-common
apt install maven build-essential cmake libssl-dev libre2-dev libcurl4-openssl-dev clang lldb lld libz-dev git ninja-build uuid-dev autoconf-archive curl zip unzip tar pkg-config bison libtool flex vim
apt install sudo
apt purge libjemalloc-dev libjemalloc2 librust-jemalloc-sys-dev
apt install -y openjdk-8-jdk
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export PATH=$JAVA_HOME/bin:$PATH
/spark/spark/bin/spark-shell --name run_gluten \
--master local --deploy-mode client \
--conf spark.plugins=io.glutenproject.GlutenPlugin \
--conf spark.memory.offHeap.enabled=true \
--conf spark.memory.offHeap.size=20g \
--jars /spark/gluten-velox-bundle-spark3.2_2.12-1.1.1.jar \
--conf spark.shuffle.manager=org.apache.spark.shuffle.sort.ColumnarShuffleManager
```
**error message**
```
root@8c55a15ec775:/spark/spark# /spark/spark/bin/spark-shell --name run_gluten \
--master local --deploy-mode client \
--conf spark.plugins=io.glutenproject.GlutenPlugin \
--conf spark.memory.offHeap.enabled=true \
--conf spark.memory.offHeap.size=20g \
--jars /spark/gluten-velox-bundle-spark3.2_2.12-1.1.1.jar \
--conf spark.shuffle.manager=org.apache.spark.shuffle.sort.ColumnarShuffleManager
24/06/14 01:39:04 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
OpenJDK 64-Bit Server VM warning: You have loaded library /tmp/gluten-7cca4e91-463a-4240-b9e4-36f67b147160/jni/11f04b97-1abd-4656-aeeb-0bedfe813394/gluten-984801035636929938/libvelox.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'.
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGILL (0x4) at pc=0x00007f2841964353, pid=42017, tid=0x00007f28da678640
#
# JRE version: OpenJDK Runtime Environment (8.0_412-b08) (build 1.8.0_412-8u412-ga-1~22.04.1-b08)
# Java VM: OpenJDK 64-Bit Server VM (25.412-b08 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C [libgluten.so+0x317353] gluten::Runtime::registerFactory(std::string const&, std::function, std::equal_to, std::allocator > > const&)>)+0x23
#
# Core dump written. Default location: /spark/spark/core or core.42017
#
# An error report file with more information is saved as:
# /spark/spark/hs_err_pid42017.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
/spark/spark/bin/spark-shell: line 47: 42017 Aborted (core dumped) "${SPARK_HOME}"/bin/spark-submit --class org.apache.spark.repl.Main --name "Spark shell" "$@"
```
What's the problem?
### System information
server: centos7
container: ubuntu22.04
### CMake log
_No response_
Contributor guide
Research direction
Reproduce the failure with the provided Ubuntu 22.04 container and Spark shell command, then inspect /spark/spark/hs_err_pid42017.log and the generated core dump. Start at the reported libgluten.so frame in gluten::Runtime::registerFactory; done means the native startup path no longer aborts with SIGILL and the cause is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cmake, docker, java, scala
- Domain
- backend, build-system, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100