microsoft / microsoft/onnxruntime

[Feature Request] C-API: Allow access to input tensor data without use of const_cast in custom op

Open
#14,917 5 comments 0 reactions 0 assignees View on GitHub
api feature request
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the feature request

Reading inputs in a custom operator through the C-API involves first getting an `OrtValue` using the following function as defined in the [public onnxruntime_c_api.h file](https://github.com/microsoft/onnxruntime/blob/f88b97ede272e54d67e7f4da165359df4b534a4a/include/onnxruntime/core/session/onnxruntime_c_api.h#L1752-L1753):

```c
ORT_API2_STATUS(
KernelContext_GetInput, _In_ const OrtKernelContext* context, _In_ size_t index, _Out_ const OrtValue** out
);
```
The retrieved `OrtValue` is `const`. As far as I can see, the only function to then read the tensor data is:

```c
ORT_API2_STATUS(GetTensorMutableData, _In_ OrtValue* value, _Outptr_ void** out);
```
which requires a non-const `OrtValue`.

For the c++ API this is ["fixed"](https://github.com/microsoft/onnxruntime/blob/f88b97ede272e54d67e7f4da165359df4b534a4a/include/onnxruntime/core/session/onnxruntime_cxx_inline.h#L1748-L1753) with a `const_cast`:
```c++
template
inline const T* CustomOpApi::GetTensorData(_Inout_ const OrtValue* value) {
T* data = nullptr;
Ort::ThrowOnError(api_.GetTensorMutableData(const_cast(value), reinterpret_cast(&data)));
return data;
}
```

It is unfortunate that the current C-API necessitates casting `const` away (as far as I can tell) in order to get to the data. Either `KernelContext_GetInput` should not enforce `const`ness or a new function should be made available in the C-API to retrieve the tensor data from a `const OrtValue*` without the risk of undefined behavior.

### Describe scenario use case

An updated C-API would make it easier and less anxiety-inducing to write custom operators. It is most likely unclear to an author of a custom op what is happening inside `GetTensorMutableData` and casting `const` away can potentially introduce undefined behavior.

Contributor guide

Open the contributing guide

Research direction

Start with the KernelContext_GetInput and GetTensorMutableData declarations in include/onnxruntime/core/session/onnxruntime_c_api.h, then compare the const handling in include/onnxruntime/core/session/onnxruntime_cxx_inline.h. Determine the C-API change needed to read input tensor data without casting away const, and verify that the resulting interface preserves safe custom-operator access.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, cpp
Domain
api, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.