NVIDIA / NVIDIA/cudf

[PERF]: Remove 1.5 GB cap from default target partition size

Open
#23,549 0 comments 0 reactions 0 assignees View on GitHub
cudf-polars Performance
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

The default target partition size is computed as:

```python
min(max(int(device_memory * 0.025), 1), 1_500_000_000)
```

The `min(..., 1.5 GB)` cap limits the partition size on GPUs with more than `1.5GB / 0.025 = 60GB` of VRAM. On a 96 GB GPU the formula gives 2.4 GB but is capped to 1.5 GB. In a sweep at SF300 on a g7e.8xlarge, we found 3.6 GB was the best partition size. We're leaving perf on the table with this conservative cap.

One option to would be be more aggressive and remove the cap so the calculation becomes
```python
max(int(min_device_size * 0.025), 1)
```
We could even remove the cap but __only__ for remote I/O.

Another option would be to bump the cap. But to what?

Contributor guide

Open the contributing guide

Research direction

Locate the code that computes the default target partition size and inspect how device memory and remote I/O are handled. Compare the proposed uncapped and higher-cap approaches using the SF300 sweep context on a g7e.8xlarge; done means reaching an agreed calculation and validating its performance impact.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
data, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.