microsoft / microsoft/onnxruntime

[Feature Request] Expose Arena initial chunk size config for cuda provider in python API

Open
#14,968 4 comments 0 reactions 0 assignees View on GitHub
ep:CUDA feature request
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the feature request

I want to set the initial arena chunk size of cuda EP. But it is not available in python API.

In C/C++ API, user could set the `OrtArenaCfg* default_memory_arena_cfg` in OrtCUDAProviderOptionsV2. Could we expose it to python API.

### Describe scenario use case

In Cuda graph, it is not allowed to call cudamalloc to extend arena memory. One way to ensure it is to allocate enough memory before graph capture. See https://github.com/microsoft/onnxruntime/issues/14942.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the C/C++ OrtCUDAProviderOptionsV2 configuration and the corresponding Python API bindings; the payload does not name specific files or tests. Confirm how default_memory_arena_cfg is represented, expose the initial arena chunk size through Python, and verify that it can be used before CUDA graph capture without requiring cudamalloc to extend the arena.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
machine-learning, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.