microsoft / microsoft/onnxruntime

[Feature Request] C++: Create arena based allocator and have access to it

Open
#18,912 3 comments 0 reactions 0 assignees View on GitHub
feature request
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the feature request

I would like to reuse a global allocator.

With api.CreateAndRegisterAllocatorV2 I can create global allocator, but I still have no access to it.

With api.CreateAllocator I can create one and have access to it, but it requires a session as a parameter.
So, I need to create a session with some dummy graph first.

### Describe scenario use case

With an access to allocator, I can reuse it to pre-allocate tensors (not with bindings), to know current utilization of a memory, etc.

It can be a new api.GetAllocator method or api.CreateAllocator without session

Contributor guide

Open the contributing guide

Research direction

Start by tracing the api.CreateAndRegisterAllocatorV2 and api.CreateAllocator entry points and how allocator access currently depends on a session. Define the API behavior needed for a reusable global allocator, including pre-allocating tensors and inspecting utilization. Done means the chosen API provides access without requiring a dummy graph or session and is covered by relevant tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.