microsoft / microsoft/onnxruntime
[Feature Request] C-API: Allow access to input tensor data without use of const_cast in custom op
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the feature request
Reading inputs in a custom operator through the C-API involves first getting an `OrtValue` using the following function as defined in the [public onnxruntime_c_api.h file](https://github.com/microsoft/onnxruntime/blob/f88b97ede272e54d67e7f4da165359df4b534a4a/include/onnxruntime/core/session/onnxruntime_c_api.h#L1752-L1753):
```c
ORT_API2_STATUS(
KernelContext_GetInput, _In_ const OrtKernelContext* context, _In_ size_t index, _Out_ const OrtValue** out
);
```
The retrieved `OrtValue` is `const`. As far as I can see, the only function to then read the tensor data is:
```c
ORT_API2_STATUS(GetTensorMutableData, _In_ OrtValue* value, _Outptr_ void** out);
```
which requires a non-const `OrtValue`.
For the c++ API this is ["fixed"](https://github.com/microsoft/onnxruntime/blob/f88b97ede272e54d67e7f4da165359df4b534a4a/include/onnxruntime/core/session/onnxruntime_cxx_inline.h#L1748-L1753) with a `const_cast`:
```c++
template
inline const T* CustomOpApi::GetTensorData(_Inout_ const OrtValue* value) {
T* data = nullptr;
Ort::ThrowOnError(api_.GetTensorMutableData(const_cast(value), reinterpret_cast(&data)));
return data;
}
```
It is unfortunate that the current C-API necessitates casting `const` away (as far as I can tell) in order to get to the data. Either `KernelContext_GetInput` should not enforce `const`ness or a new function should be made available in the C-API to retrieve the tensor data from a `const OrtValue*` without the risk of undefined behavior.
### Describe scenario use case
An updated C-API would make it easier and less anxiety-inducing to write custom operators. It is most likely unclear to an author of a custom op what is happening inside `GetTensorMutableData` and casting `const` away can potentially introduce undefined behavior.
Contributor guide
Research direction
Start with the KernelContext_GetInput and GetTensorMutableData declarations in include/onnxruntime/core/session/onnxruntime_c_api.h, then compare the const handling in include/onnxruntime/core/session/onnxruntime_cxx_inline.h. Determine the C-API change needed to read input tensor data without casting away const, and verify that the resulting interface preserves safe custom-operator access.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, cpp
- Domain
- api, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100