ODAPI failed to initiate by running into runtime error.

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• [x ] I am using the latest TensorFlow Model Garden release and TensorFlow 2.
• I am reporting the issue to the correct repository. (Model Garden official or research directory)
• [x ] I checked to make sure that this issue has not already been filed.

1. The entire URL of the file you are using

https://github.com/tensorflow/models/blob/master/research/object_detection/model_main_tf2.py

2. Describe the bug

The ODAPI failed to initiate by running into runtime error.

Traceback (most recent call last):
File "/content/models/research/object_detection/model_main_tf2.py", line 113, in
tf.compat.v1.app.run()
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 300, in run
_run_main(main, args)
File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 251, in _run_main
sys.exit(main(argv))
File "/content/models/research/object_detection/model_main_tf2.py", line 110, in main
record_summaries=FLAGS.record_summaries)
File "/usr/local/lib/python3.6/dist-packages/object_detection/model_lib_v2.py", line 523, in train_loop
dummy_prediction_dict = detection_model.predict(dummy_image, dummy_shapes)
File "/usr/local/lib/python3.6/dist-packages/object_detection/meta_architectures/faster_rcnn_meta_arch.py", line 830, in predict
**side_inputs))
File "/usr/local/lib/python3.6/dist-packages/object_detection/meta_architectures/faster_rcnn_meta_arch.py", line 999, in _predict_second_stage
image_shape, true_image_shapes)
File "/usr/local/lib/python3.6/dist-packages/object_detection/meta_architectures/faster_rcnn_meta_arch.py", line 729, in _proposal_postprocess
anchors, image_shape_2d, true_image_shapes)
File "/usr/local/lib/python3.6/dist-packages/object_detection/meta_architectures/faster_rcnn_meta_arch.py", line 1754, in _postprocess_rpn
) = self._format_groundtruth_data(image_shapes)
File "/usr/local/lib/python3.6/dist-packages/object_detection/meta_architectures/faster_rcnn_meta_arch.py", line 1891, in _format_groundtruth_data
self.groundtruth_lists(fields.BoxListFields.boxes))
File "/usr/local/lib/python3.6/dist-packages/object_detection/core/model.py", line 118, in groundtruth_lists
field))
RuntimeError: Groundtruth tensor boxes has not been provided

3. Steps to reproduce

Clone the latest model, run model_main_tf2.py by Faster R-CNN ResNet101 V1 800x1333 config.

4. Expected behaviour

The training initiate as smooth as if before.

5. Additional context

Revert to the last model_lib_v2.py before EMA changes solved the problem.

6. System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Google Colab
• Mobile device name if the issue happens on a mobile device:
• TensorFlow installed from (source or binary): source
• TensorFlow version (use command below): 2.4
• Python version: 3.6
• Bazel version (if compiling from source):
• GCC/Compiler version (if compiling from source):
• CUDA/cuDNN version: cuda 10.1 cudnn 7.0
• GPU model and memory: P100 16gb