microsoft / microsoft/onnxruntime
IOBinding for Tensorflow Tensors
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 8h
- Merged PRs (30d)
- 179
Description
Is your feature request related to a problem? Please describe.
I am running some tf.data.Dataset data-pipelines where I've got tf.Tensor's inside.
I am predicting with ONNX because of the very convenient ExecutionProvider mechanism. Most of the time I am using TensorRT Execution Provier.
But predicting with ONNX means I have to copy the tf.Tensor to Host (D2H, assuming Tensor is located on device), supply it to the ONNX session (H2D->predict->D2H) and put it back to device (H2D, if required).
So in the worst case the data will be copied 4 times.
System information
- onnx 1.11.0
- onnxconverter-common 1.9.0
- onnxruntime-gpu 1.11.1
Describe the solution you'd like
Make io_binding.bind_input() compatibile with tf.Tensor
Additional context
I am politely asking because I think tf and torch are great frameworks and the work for compatibility might be worth it :-).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the io_binding.bind_input() entry point and trace how inputs are accepted and transferred. The issue names no files or tests; done means tf.Tensor inputs are accepted while avoiding unnecessary host-device copies.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- tensorflow
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100