microsoft / microsoft/onnxruntime

[Documentation] Lifetime of device tensors and allocating device tensors with the env allocator

Open
#24,007 8 comments 0 reactions 1 assignee Claimed by @tianleiwu View on GitHub
documentation
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the documentation issue

When calling [CreateAndRegisterAllocatorV2](https://onnxruntime.ai/docs/api/c/struct_ort_api.html#a12c97c7471565c34f99dac7e574d12cf) is there any way to use the created allocator without an OrtSession ?

A flow that should work is to register the env allocator, create a Session and then acquire the arena allocator (should be the env allocator at this point) through the API shown here. https://onnxruntime.ai/docs/performance/device-tensor.html#cuda

I am thinking about the lifetime of device tensors when using the session allocator, ideally I would syntactically be able to acquire the environment allocator somehow from the environment. That would make it clear that the allocators lifetime is associated with the environment and not just with the session.

### Page / URL

https://onnxruntime.ai/docs/get-started/with-c.html#features

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.