microsoft / microsoft/onnxruntime
[Feature Request] C-API: Allow access to input string tensors without copying
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the feature request
The current C-API performs a copy of input string tensors rather than allowing access to the existing data by reference. The relevant functions are [`GetStringTensorContent`](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/session/onnxruntime_c_api.h#L1309) and [`GetStringTensorElement`](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/session/onnxruntime_c_api.h#L2132). Either of those functions require the user to allocate buffers into which the requested strings are copied.
It would be great to have a way to access string inputs without the copy similar to how [`GetTensorMutableData`](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/session/onnxruntime_c_api.h#L1267) does it for numerical data. The precise solution would depend on how the data is stored on the `onnxruntime` side, but I have not been able to determine that. Any insights to that would be much appreciated! It might be that the data is fragmented. If so, at least an element-wise non-copy access should hopefully be feasible.
### Describe scenario use case
Avoiding the copy would allow for better performance and memory usage in custom operators.
Contributor guide
Research direction
Start with include/onnxruntime/core/session/onnxruntime_c_api.h, especially GetStringTensorContent, GetStringTensorElement, and GetTensorMutableData, then trace how string tensor data is stored. Done would be a defined C-API approach that exposes input strings without copying, including an element-wise option if contiguous access is not possible, with validation for custom-operator use.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, cpp
- Domain
- api, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100