microsoft / microsoft/onnxruntime

[Feature Request] C-API: Allow access to input string tensors without copying

Open
#14,918 7 comments 0 reactions 0 assignees View on GitHub
api feature request
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the feature request

The current C-API performs a copy of input string tensors rather than allowing access to the existing data by reference. The relevant functions are [`GetStringTensorContent`](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/session/onnxruntime_c_api.h#L1309) and [`GetStringTensorElement`](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/session/onnxruntime_c_api.h#L2132). Either of those functions require the user to allocate buffers into which the requested strings are copied.

It would be great to have a way to access string inputs without the copy similar to how [`GetTensorMutableData`](https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/session/onnxruntime_c_api.h#L1267) does it for numerical data. The precise solution would depend on how the data is stored on the `onnxruntime` side, but I have not been able to determine that. Any insights to that would be much appreciated! It might be that the data is fragmented. If so, at least an element-wise non-copy access should hopefully be feasible.

### Describe scenario use case

Avoiding the copy would allow for better performance and memory usage in custom operators.

Contributor guide

Open the contributing guide

Research direction

Start with include/onnxruntime/core/session/onnxruntime_c_api.h, especially GetStringTensorContent, GetStringTensorElement, and GetTensorMutableData, then trace how string tensor data is stored. Done would be a defined C-API approach that exposes input strings without copying, including an element-wise option if contiguous access is not possible, with validation for custom-operator use.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, cpp
Domain
api, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.