[PERF]: kvikio byte range prefetch backend for hybrid scan
Open
cudf-polars
Performance
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
Byte range prefetch backend using kvikio's pread API to fetch data into pinned host memory ahead of evaluation. kvikio supports two remote IO backends: EASY_THREADPOOL (default, thread pool based) and MULTI_POLL (reactor based, similar to cuCascade). We need to evaluate both.
PR: #23317
Contributor guide
Research direction
Start by reading PR #23317 and the kvikio pread API details in the issue. Compare the EASY_THREADPOOL and MULTI_POLL remote I/O backends for byte-range prefetch into pinned host memory, and document or implement the evaluation for hybrid scan once both approaches have been assessed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- data-engineering, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100