tensorflow / tensorflow/models
TF2 exporter doesn't support dynamic batch size
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.7k
- Forks
- 44.8k
- PR merge metrics
- No merged PRs in 30d
Description
1. The entire URL of the file you are using
https://github.com/tensorflow/models/blob/master/research/object_detection/exporter_main_v2.py
2. Describe the bug
exporter_main_v2.py does not support changing the input shape. This feature was added to the tf1 exporter (export_inference_graph.py) in #2053, but looks like it's not included in the current tf2 version. As a result, all exported saved models have a fixed batch size of 1. In order to enable dynamic batching in triton inference server, we need an dynamic batch size. Please see related [issue] and recommendation for fix on triton github (https://github.com/triton-inference-server/server/issues/2097)
3. Steps to reproduce
Follow tutorial on exporting saved model https://tensorflow-object-detection-api-tutorial.readthedocs.io/en/latest/training.html#exporting-a-trained-model
4. Expected behavior
exporter_main_v2.py should accept an input_shape argument and default to [None, None, None, 3] such that exported models support dynamic batching, similar to tf1 export_inference_graph.py
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.