NVIDIA / NVIDIA/cudf

[PERF]: Enable pinned memory by default

Open
#23,537 2 comments 0 reactions 0 assignees View on GitHub
cudf-polars Performance
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

Byte range prefetching requires pinned host memory. Pinned memory is currently off by default because the initial pool allocation is slow. We should investigate how long this takes and think of some way to start the intialization of the pool as early as possible. And reuse it for subsequent queries or iterations of the the same query.

Contributor guide

Open the contributing guide

Research direction

No file or test is identified in the issue. Start by locating the pinned host-memory pool and byte-range prefetch initialization, then measure the initial allocation cost and its timing across repeated queries or iterations; done means the measured behavior supports an agreed approach for earlier initialization and reuse.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
hpc, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.