libvelox.so issue while running Spark tpcds/tpch workload with Gluten
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Backend
VL (Velox)
### Bug description
Running on GCP EMR 80 vcpu with Gluten enable ended up below issue . No issue on N2 - ICX 16 VCPU run .
Same issue on both TPCH and TPCDS run : NO Issue to run same code in Vanilla . Using Spark notebook to execute this :
Run parameters below :
### Spark version
Spark-3.3.x
### Spark configurations
Standard Gluten configurations - run file on ICX GCP N2 machine
### System information
Ubuntu 20.04 using the Gluten Velox jar
### Relevant logs
_No response_
Contributor guide
Research direction
The report names a Spark notebook, TPCH and TPCDS workloads, and the Gluten Velox jar, but no source files or tests. Start by reproducing the libvelox.so failure on GCP EMR and comparing it with the working N2 ICX run, while collecting the missing logs. Done requires a reproducible diagnosis of the environment-specific failure and a verified fix or actionable next step.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- gcp, scala, spark, ubuntu
- Domain
- backend, data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100