microsoft / microsoft/onnxruntime
Documentation for io binding
Open
@natke is already working on this.
Since Apr 14, 2022.
documentation
feature request
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 8h
- Merged PRs (30d)
- 179
Description
Is your feature request related to a problem? Please describe.
I want to bind some inputs/outputs to gpu (output of a model to be passed to another model as input).
The documentation and examples for io binding in C# are lacking. I have found these two tests here and here.
The most information is in the C header file here but onnxruntime_c_api.h header was confusing.
It is still unclear what these functions do, and when they are needed.
For example
- Do we need to call SynchronizeBoundInputs before
session.runand then SynchronizeBoundOutputs after run? - Do we need to call the BindInput each time before
session.runor just once and then call SynchronizeBoundInputs? - What happens if the bound input memory values change, do we need to call the BindInput again, or just SynchronizeBoundInputs?
- I have an input (that changes), but has a fixed size (even though the model could handle dynamic batch size). Does that count as fixedbuffer? do I need to call BindInput each time?
- There is an output that I would like to keep on GPU. But the rest of input and outputs I do not care to bind. Still for that one input I need to change the API call for everything, because there is no method to just bind a single output to device
System information
- ONNX Runtime version (you are using): 1.10
Describe the solution you'd like
Documentation and examples for C#
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.