Export Object detection model (V2) fails on assertion "assert_existing_objects_matched"

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• [+] I am using the latest TensorFlow Model Garden release and TensorFlow 2.
• [+] I am reporting the issue to the correct repository. (Model Garden official or research directory)
• [+] I checked to make sure that this issue has not been filed already.

1. The entire URL of the file you are using

https://github.com/tensorflow/models/tree/master/research/object_detection

2. Describe the bug

I'm having the OOM issue on the big models, so I tried to train a dummy model,

faster_rcnn {
    num_classes: 9
    image_resizer {
      keep_aspect_ratio_resizer {
        min_dimension: 80
        max_dimension: 100
        pad_to_max_dimension: true
      }
    }
...
train_config: {
  batch_size: 1
  num_steps: 200
....

for a some reason resulting checkpoint files are very small
ckpt-1.index = 247 bytes
ckpt-1.data-00000-of-00001 = 864 bytes

Export of this dummy model fails with assertion

raise AssertionError(
("Some Python objects were not bound to checkpointed values, likely due to changes in the Python program: %s") %
(list(unused_python_objects),))

3. Steps to reproduce

Train dummy faster_rcnn model without finetune checkpoint

4. Expected behavior

final checkpoint file has to be ~100 Mb like in v1. Export happens without errors

5. Additional context

6. System information

• Linux Ubuntu 18
• TensorFlow installed from binary
• TensorFlow version v2.2.0-rc4-8-g2b96f3662b 2.2.0
• Python version 3.8
• CUDA/cuDNN version: Driver Version: 440.100 CUDA Version: 10.2
• GPU model and memory: GeForce GTX 960M / 2004MiB