apache / apache/gluten

[VL] Support rss_sort shuffle write for uniffle

Open
#11,551 0 comments 0 reactions 1 assignee Claimed by @wForget View on GitHub
enhancement
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Description

As mentioned in https://github.com/apache/incubator-gluten/issues/11542#issuecomment-3839521996 , there is a memory threshold limit in `rss_sort` shuffle write, which helps avoid OOM errors. Therefore, I'd like to support `rss_sort` shuffle write for Uniffle.

### Gluten version

main branch

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.