Number of checkpoints saved do not match the checkpoints per step flag

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• [ x] I am using the latest TensorFlow Model Garden release and TensorFlow 2.
• [x ] I am reporting the issue to the correct repository. (Model Garden official or research directory)
• [ x] I checked to make sure that this issue has not already been filed.

1. The entire URL of the file you are using

https://github.com/tensorflow/models/tree/master/research/object_detection/model_main_tf2.py

2. Describe the bug

Checkpoints saved do not match the number defined in training, it stores n+1 checkpoints than expected, where n is the number of checkpoints specified.

3. Steps to reproduce

python model_main_tf2.py --
--model_dir=$MODEL_DIR --num_train_steps=$NUM_TRAIN_STEPS
--sample_1_of_n_eval_examples=$SAMPLE_1_OF_N_EVAL_EXAMPLES
--pipeline_config_path=$PIPELINE_CONFIG_PATH
--checkpoint_every_n=100

4. Expected behavior

I must expect that given a number of steps reached in training, lets say 1500, I'd have 15 checkpoints. Nevertheless I see 16 checkpoints saved.

5. Additional context

Here I have 1500 steps, and when executing the script specified to store checkpoints every 100 steps.

But, when I chek my checkpoints folder it shows 16 checkpoints. So I'm looking to export a model from 14th checkpoint, but what checkpoint should I choose, 13th, 14th or 15th?

6. System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 20.04
• Mobile device name if the issue happens on a mobile device:
• TensorFlow installed from (source or binary): binary
• TensorFlow version (use command below):
• Python version: 2.2.0
• Bazel version (if compiling from source):
• GCC/Compiler version (if compiling from source):
• CUDA/cuDNN version: 11.0, 8.1
• GPU model and memory: Nvidia GTX 1660 Ti 6GB