apache / apache/gluten

[VL] JVM error on facebook::velox::cache::AsyncDataCache::toString[abi:cxx11](bool)

Open
#3,755 1 comment 1 reaction 0 assignees View on GitHub
bug
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Backend

VL (Velox)

### Bug description

Report a bug per Weiting's recomendation.

Steps to produce
1. Set up spark to run with gluten.
2. Run this script with `./run-etl-tpch.sh 0 1` -> https://github.com/intel-sandbox/tools.hyperscalers.infra.aws.dataservice.utility/blob/master/etl/spark/tpch/run-etl-tpch.sh.
3. py script -> https://github.com/intel-sandbox/tools.hyperscalers.infra.aws.dataservice.utility/blob/master/etl/spark/tpch/spark-tpch-csv-2-parquet.py
4. ETL job was completed, shutdown hooks were called. But at the very end, an error message occurred. Message and screenshot below.
Java VM: OpenJDK 64-Bit Server VM (25.382-b05 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C [libvelox.so+0x531b97f] facebook::velox::cache::AsyncDataCache::toString[abi:cxx11](bool) const+0x2af
![image](https://github.com/oap-project/gluten/assets/99760781/7b94e0ee-351a-46bf-a3b6-3f2da8dd58a4)

Additional information
spark logs -> http://aws-spr5.ra.intel.com:8080/history/application_1699027314997_0101/jobs/

### Spark version

Spark-3.2.x

### Spark configurations

Compelete spark env -> http://aws-spr5.ra.intel.com:8080/history/application_1699027314997_0101/environment/

spark-defaults.conf below

spark.plugins io.glutenproject.GlutenPlugin
spark.gluten.sql.native.writer.enabled true
spark.gluten.sql.columnar.backend.lib velox
spark.gluten.sql.columnar.shuffle.batchCompressThreshold 0
spark.gluten.sql.columnar.forceShuffledHashJoin true
spark.shuffle.manager org.apache.spark.shuffle.sort.ColumnarShuffleManager
spark.gluten.sql.columnar.backend.velox.cacheEnabled true
spark.gluten.sql.columnar.backend.velox.memCacheSize 3221225472
spark.gluten.sql.columnar.backend.velox.ssdCacheSize 3221225472
spark.sql.parquet.compression.codec zstd
spark.memory.offHeap.enabled true
spark.memory.offHeap.size 10g
spark.executor.memoryOverhead 1g
spark.driver.maxResultSize 2g
spark.sql.shuffle.partitions 256
spark.eventLog.enabled true
spark.eventLog.dir file:///home/ubuntu/sparkEventLog
spark.history.fs.logDirectory file:///home/ubuntu/sparkEventLog
spark.history.provider org.apache.spark.deploy.history.FsHistoryProvider
spark.history.fs.update.interval 10s
spark.history.ui.port 8080

### System information

_No response_

### Relevant logs

```bash
spark logs -> http://aws-spr5.ra.intel.com:8080/history/application_1699027314997_0101/jobs/
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.