tensorflow / tensorflow/models

TF2 exporter doesn't support dynamic batch size

Open
#9,358 14 comments 1 reaction 3 assignees View on GitHub

Nobody has claimed this yet.

models:research:odapi type:bug
Dominant language
Python
Stars
77.7k
Forks
44.8k
PR merge metrics
No merged PRs in 30d

Description

1. The entire URL of the file you are using

https://github.com/tensorflow/models/blob/master/research/object_detection/exporter_main_v2.py

2. Describe the bug

exporter_main_v2.py does not support changing the input shape. This feature was added to the tf1 exporter (export_inference_graph.py) in #2053, but looks like it's not included in the current tf2 version. As a result, all exported saved models have a fixed batch size of 1. In order to enable dynamic batching in triton inference server, we need an dynamic batch size. Please see related [issue] and recommendation for fix on triton github (https://github.com/triton-inference-server/server/issues/2097)

3. Steps to reproduce

Follow tutorial on exporting saved model https://tensorflow-object-detection-api-tutorial.readthedocs.io/en/latest/training.html#exporting-a-trained-model

4. Expected behavior

exporter_main_v2.py should accept an input_shape argument and default to [None, None, None, 3] such that exported models support dynamic batching, similar to tf1 export_inference_graph.py

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.