[VL] One case of should spill but OOM
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Backend
VL (Velox)
### Bug description
I suppose in below case, UnsafeExternalSorter should spill.
```
Caused by: java.lang.RuntimeException: Error during calling Java code from native code: io.glutenproject.memory.memtarget.ThrowOnOomMemoryTarget$OutOfMemoryException: Not enough spark off-heap execution memory. Acquired: 8388608, granted: 8126464. Try tweaking config option spark.memory.offHeap.size to get larger space to run this application.
Current config settings:
spark.gluten.memory.offHeap.size.in.bytes=12.0 GiB
spark.gluten.memory.task.offHeap.size.in.bytes=3.0 GiB
spark.gluten.memory.conservative.task.offHeap.size.in.bytes=1536.0 MiB
Memory consumer stats:
Task.241798: Current used bytes: 3.0 GiB, peak bytes: N/A
+- org.apache.spark.util.collection.unsafe.sort.UnsafeExternalSorter@4479ef33: Current used bytes: 2.1 GiB, peak bytes: N/A
\- Gluten.Tree.18: Current used bytes: 952.0 MiB, peak bytes: 1048.0 MiB
\- root.18: Current used bytes: 952.0 MiB, peak bytes: 1048.0 MiB
+- ShuffleReader.18: Current used bytes: 560.0 MiB, peak bytes: 600.0 MiB
| \- single: Current used bytes: 256.0 MiB, peak bytes: 272.0 MiB
| \- ShuffleReader_default_leaf: Current used bytes: 248.1 MiB, peak bytes: 270.3 MiB
+- ColumnarToRow.18: Current used bytes: 384.0 MiB, peak bytes: 656.0 MiB
| \- single: Current used bytes: 384.0 MiB, peak bytes: 640.0 MiB
| \- ColumnarToRow_default_leaf: Current used bytes: 384.0 MiB, peak bytes: 640.0 MiB
+- ArrowContextInstance.18: Current used bytes: 8.0 MiB, peak bytes: 8.0 MiB
+- OverAcquire.DummyTarget.52: Current used bytes: 0.0 B, peak bytes: 0.0 B
+- WholeStageIterator.18: Current used bytes: 0.0 B, peak bytes: 0.0 B
| \- single: Current used bytes: 0.0 B, peak bytes: 0.0 B
| +- WholeStageIterator_default_leaf: Current used bytes: 0.0 B, peak bytes: 0.0 B
| \- task.Gluten_Stage_13_TID_241798: Current used bytes: 0.0 B, peak bytes: 0.0 B
| +- node.1: Current used bytes: 0.0 B, peak bytes: 0.0 B
| | \- op.1.0.0.FilterProject: Current used bytes: 0.0 B, peak bytes: 0.0 B
| \- node.0: Current used bytes: 0.0 B, peak bytes: 0.0 B
| \- op.0.0.0.ValueStream: Current used bytes: 0.0 B, peak bytes: 0.0 B
+- OverAcquire.DummyTarget.56: Current used bytes: 0.0 B, peak bytes: 0.0 B
\- OverAcquire.DummyTarget.55: Current used bytes: 0.0 B, peak bytes: 0.0 B
```
### Spark version
None
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
_No response_
Contributor guide
Research direction
Start by tracing Spark's UnsafeExternalSorter interaction with Gluten's ThrowOnOomMemoryTarget and the Velox memory consumers shown in the report. Reconstruct the missing Spark configuration and workload details, then verify why the reported allocation is not followed by spilling; done means the case no longer fails with an out-of-memory error or is shown to be configuration-related.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100