microsoft / microsoft/onnxruntime

Documentation for io binding

Open
#11,133 5 comments 1 reaction 1 assignee View on GitHub

@natke is already working on this.

Since Apr 14, 2022.

documentation feature request
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 8h
Merged PRs (30d)
179

Description

Is your feature request related to a problem? Please describe.
I want to bind some inputs/outputs to gpu (output of a model to be passed to another model as input).
The documentation and examples for io binding in C# are lacking. I have found these two tests here and here.
The most information is in the C header file here but onnxruntime_c_api.h header was confusing.

It is still unclear what these functions do, and when they are needed.
For example

  1. Do we need to call SynchronizeBoundInputs before session.run and then SynchronizeBoundOutputs after run?
  2. Do we need to call the BindInput each time before session.run or just once and then call SynchronizeBoundInputs?
  3. What happens if the bound input memory values change, do we need to call the BindInput again, or just SynchronizeBoundInputs?
  4. I have an input (that changes), but has a fixed size (even though the model could handle dynamic batch size). Does that count as fixedbuffer? do I need to call BindInput each time?
  5. There is an output that I would like to keep on GPU. But the rest of input and outputs I do not care to bind. Still for that one input I need to change the API call for everything, because there is no method to just bind a single output to device

System information

  • ONNX Runtime version (you are using): 1.10

Describe the solution you'd like
Documentation and examples for C#

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.