microsoft / microsoft/onnxruntime
[Feature Request] Expose Arena initial chunk size config for cuda provider in python API
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the feature request
I want to set the initial arena chunk size of cuda EP. But it is not available in python API.
In C/C++ API, user could set the `OrtArenaCfg* default_memory_arena_cfg` in OrtCUDAProviderOptionsV2. Could we expose it to python API.
### Describe scenario use case
In Cuda graph, it is not allowed to call cudamalloc to extend arena memory. One way to ensure it is to allocate enough memory before graph capture. See https://github.com/microsoft/onnxruntime/issues/14942.
Contributor guide
Research direction
Start by tracing the C/C++ OrtCUDAProviderOptionsV2 configuration and the corresponding Python API bindings; the payload does not name specific files or tests. Confirm how default_memory_arena_cfg is represented, expose the initial arena chunk size through Python, and verify that it can be used before CUDA graph capture without requiring cudamalloc to extend the arena.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- machine-learning, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100