apache / apache/gluten

[VL] Enhance Velox Cache to Support Relative Paths and Multiple Disk Storage

Open
#5,205 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Description

Currently, Velox Cache only supports absolute file paths. It would be beneficial to extend support to relative paths as well, allowing users to specify paths relative to the spark.local.dirs directory. Additionally, enabling Velox Cache to utilize multiple disk storage would enhance its versatility and suitability for various use cases.

The current PR depends on [#9284](https://github.com/facebookincubator/velox/pull/9284) in velox, which needs to be merged before it can be supported.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.