[VL] Dynamic Offheap Sizing Feature Returns False OOM When SQL Conf is Empty
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Backend
VL (Velox)
### Bug description
```
data = [{"col1": 1}, {"col1": 2}, {"col1": 3}, {"col1": 4}]
df=spark.createDataFrame(data)
df.select("col1").distinct().rdd.flatMap(lambda x:x).collect()
```
Returns
```
2025-03-12 10:45:37,789 ERROR ManagedReservationListener [Executor task launch worker for task 6.0 in stage 3.0 (TID 15)]: Error reserving memory from target
org.apache.gluten.memory.memtarget.ThrowOnOomMemoryTarget$OutOfMemoryException: Not enough spark off-heap execution memory. Acquired: 8.0 MiB, granted: 0.0 B. Try tweaking config option spark.memory.offHeap.size to get larger space to run this application (if spark.gluten.memory.dynamic.offHeap.sizing.enabled is not enabled).
Current config settings:
spark.gluten.memory.offHeap.size.in.bytes=0.0 B
spark.memory.offHeap.size=0.0 B
spark.gluten.memory.task.offHeap.size.in.bytes=0.0 B
spark.gluten.memory.conservative.task.offHeap.size.in.bytes=0.0 B
spark.memory.offHeap.enabled=false
spark.gluten.memory.dynamic.offHeap.sizing.enabled=false
Memory consumer stats:
Task.15: Current used bytes: 0.0 B, peak bytes: N/A
\- Gluten.Tree.2: Current used bytes: 0.0 B, peak bytes: 0.0 B
\- root.2: Current used bytes: 0.0 B, peak bytes: 0.0 B
+- ShuffleWriter.2: Current used bytes: 0.0 B, peak bytes: 0.0 B
\- OverAcquire.DummyTarget.5: Current used bytes: 0.0 B, peak bytes: 0.0 B
at org.apache.gluten.memory.memtarget.ThrowOnOomMemoryTarget.borrow(ThrowOnOomMemoryTarget.java:114)
at org.apache.gluten.memory.listener.ManagedReservationListener.reserve(ManagedReservationListener.java:43)
at org.apache.gluten.exec.RuntimeJniWrapper.createRuntime(Native Method)
```
because SQLConf does not have these configs. It is better to get these from `SparkEnv`.
### Spark version
Spark-3.5.x
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
```bash
```
Contributor guide
Research direction
Start by tracing how SQLConf and SparkEnv provide off-heap settings to RuntimeJniWrapper.createRuntime, then reproduce the provided Spark 3.5.x distinct/RDD example with the SQL configuration empty. Done means the empty SQLConf case obtains the correct settings instead of granting 0 bytes and raising a false OOM.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala, sql
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100