[PERF]: Remove 1.5 GB cap from default target partition size
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
The default target partition size is computed as:
```python
min(max(int(device_memory * 0.025), 1), 1_500_000_000)
```
The `min(..., 1.5 GB)` cap limits the partition size on GPUs with more than `1.5GB / 0.025 = 60GB` of VRAM. On a 96 GB GPU the formula gives 2.4 GB but is capped to 1.5 GB. In a sweep at SF300 on a g7e.8xlarge, we found 3.6 GB was the best partition size. We're leaving perf on the table with this conservative cap.
One option to would be be more aggressive and remove the cap so the calculation becomes
```python
max(int(min_device_size * 0.025), 1)
```
We could even remove the cap but __only__ for remote I/O.
Another option would be to bump the cap. But to what?
Contributor guide
Research direction
Locate the code that computes the default target partition size and inspect how device memory and remote I/O are handled. Compare the proposed uncapped and higher-cap approaches using the SF300 sweep context on a g7e.8xlarge; done means reaching an agreed calculation and validating its performance impact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- data, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100