tensorflow / tensorflow/tflite-support
How to handle dynamic output tensors with `tflite.runForMultipleInputsOutputs`
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 441
- Forks
- 146
- PR merge metrics
- No merged PRs in 30d
Description
I'm working with a text to speech model so I cannot predict the size of the output tensors beforehand. According to the documentation, "some models have dynamic outputs, where the shape of output tensors can vary depending on the input. There's no straightforward way of handling this with the existing Java inference API, but planned extensions will make this possible." What's the non-straightforward way of handling it? For tflite.run, it says here that you can pass null as output but the same doesn't work with runMultipleInputsOutputs.
P.S: I cannot use a large enough buffer than can handle all possible inputs that can be passed to the model because I actually I have divided my model into 3 tflite files and I need to pass the output of one model to another model and therefore it has to be of the exact size that the model expects..
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Java inference API entry points tflite.runForMultipleInputsOutputs, tflite.run, and Tensor.asReadOnlyBuffer(), then compare how dynamic outputs are handled in each path. Determine the supported behavior or required API change for variable-sized outputs, and validate it with a focused regression test and updated documentation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100