[VL] Support GPU shuffle reader
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Description
1. Move the lock from WholeStageResultIterator constructor to shuffle reader, the threads can prepare the first batch in advance, now it is a small batch, for GPU, it will produce a big batch with bytes 1GB
2. Implement the decompression, resize batch in cpu, convert the batch to gpu when gpu is available
3. Save the batch to cache and consider the server memory to fetch the number of batches.
4. Polling threads to fetch the batch from several CPU threads and feed into GPU
5. The first batch is synchronous to fetch, may be a less size batch and processed by GPU to let GPU operate always.
6. The priority pool to fetch data from which threads(more batches, more priority)
7. The probe side batch is bigger that build side batch, and processed after build is finished, so the build threads will have high priority than probe when the memory is not enough.
### Gluten version
None
Contributor guide
Research direction
Start by tracing the WholeStageResultIterator constructor and the shuffle reader mentioned in the description. Map the proposed CPU decompression and resizing, GPU conversion, caching, polling, and priority behavior before deciding on an implementation. Done means the shuffle reader supports the listed batching, memory, threading, and build-versus-probe priority requirements.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- data-engineering, distributed-systems, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100