apache / apache/gluten

[VL] Velox memCache is invisible to Spark's memory manager

Open
#12,671 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 21h
Merged PRs (30d)
85

Description

### Description

When `spark.gluten.sql.columnar.backend.velox.cacheEnabled` is on, `VeloxBackend::initCache()`
builds an `MmapAllocator` sized from `memCacheSize` and hands it to the `AsyncDataCache`:

```cpp
options.capacity = memCacheSize;
cacheAllocator_ = std::make_shared(options);
if (ssdCacheSize == 0) {
// TODO: this is not tracked by Spark.
asyncDataCache_ = velox::cache::AsyncDataCache::create(cacheAllocator_.get());
} else {
// TODO: this is not tracked by Spark.
auto ssd = initSsdCache(ssdCacheSize);
asyncDataCache_ = velox::cache::AsyncDataCache::create(cacheAllocator_.get(), std::move(ssd));
}
```

Spark never sees that memory. The two `// TODO: this is not tracked by Spark.` comments in
`cpp/velox/compute/VeloxBackend.cc` record exactly this.

#### Why it matters

An executor ends up occupying its Spark budget **plus** the whole cache. Every decision Spark
makes from its own books is then made against a number that is short by up to the cache size:

- when to spill,
- when to fail a task with an OOM,
- how much a task may acquire.

The cache is also fixed at its configured size for the lifetime of the executor. It cannot give
anything back when queries need the memory, which is the wrong priority: a query that has to
spill costs far more than the cache misses that shrinking would cause.

#### Proposal

Make the cache capacity something Spark can move at runtime, and register what the cache occupies
with Spark's memory manager, so that:

1. the cache's footprint is visible to Spark on **all** supported Spark versions;
2. the cache yields memory when execution needs it, and reclaims it when the pressure lifts;
3. usage stays bounded by what Spark has agreed to.

An earlier attempt used Spark 4.1's `UnmanagedMemoryConsumer`, which only works on 4.1+ and reacts
only at task boundaries.

This description was written with the assistance of generative AI tooling (GitHub Copilot CLI).

### Gluten version

main branch

Contributor guide

Open the contributing guide

Research direction

Read cpp/velox/compute/VeloxBackend.cc, especially VeloxBackend::initCache(), and trace how the MmapAllocator and AsyncDataCache are created. Done means the cache footprint is visible to Spark on every supported Spark version, can yield and reclaim capacity at runtime, and remains within Spark’s agreed bound.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.