IoU localization loss function doesn't work with object detection - Wrong tensor format inside function

System information

• What is the top-level directory of the model you are using: object_detection
• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Mac OS 10.13.3
• TensorFlow installed from (source or binary): Tensorflow models installed from source, tf 1.9.0 installed from binary
• TensorFlow version (use command below): tf 1.9.0
• Bazel version (if compiling from source): N/A
• CUDA/cuDNN version: N/A
• GPU model and memory: GeForce RTX
• Exact command to reproduce: N/A

Describe the problem

One option of localization loss provided is the IOU loss.

When I first tried to use it, I got a problem with the shapes.. it reshapes the loss output to a 1D array, and this is not what the rest of the code expects, so I've reshaped to [batch_size, num_anchors, -1] and it worked, but the values were not converging.. so I went to further
investigation.

The function that calculates the IoU (matched_iou()) expects the boxes in a decoded format ([Xmin, Ymin, Xmax, Ymax]), but if you print the prediction and target tensors, they come in the encoded format (according to the default boxes). In order to decode it, you need to use the default boxes, which at this point in the code are not there anymore.

Source code / logs

Code changed by me in the IoU Localization loss class in object_detection/core/losses.py

    def _compute_loss(self, prediction_tensor, target_tensor, weights):
        """Compute loss function.

        Args:
          prediction_tensor: A float tensor of shape [batch_size, num_anchors, 4]
            representing the decoded predicted boxes
          target_tensor: A float tensor of shape [batch_size, num_anchors, 4]
            representing the decoded target boxes
          weights: a float tensor of shape [batch_size, num_anchors]

        Returns:
          loss: a float tensor of shape [batch_size, num_anchors] tensor
            representing the value of the loss function.
        """

        prediction_tensor = tf.Print(prediction_tensor, [prediction_tensor], 'Prediction tensor in IOU loss: ', summarize=8)
        target_tensor = tf.Print(target_tensor, [target_tensor], 'Target tensor in IOU loss: ', summarize=8)

        shape = prediction_tensor.get_shape().as_list()
        batch_size, num_anchors, _ = shape

        predicted_boxes = box_list.BoxList(tf.reshape(prediction_tensor, [-1, 4]))
        target_boxes = box_list.BoxList(tf.reshape(target_tensor, [-1, 4]))
        per_anchor_iou_loss = 1.0 - box_list_ops.matched_iou(predicted_boxes,
                                                             target_boxes)

        return tf.reshape(weights, [batch_size, num_anchors, -1]) * tf.reshape(per_anchor_iou_loss,\
                                                                               [batch_size, num_anchors, -1])

Output:

Prediction tensor in IOU loss: [[[0.91901803 1.04905343 0.639230669 -0.0775884241][-0.474719614 0.0712455213 -0.291270375 1.18328691]]...]
Target tensor in IOU loss: [[[0 0 0 0][0 0 0 0]]...]
Prediction tensor in IOU loss: [[[0.611483 1.27656579 0.435161799 0.46589154][-0.00451424718 0.386824429 -0.291154832 0.644339323]]...]
Target tensor in IOU loss: [[[0 0 0 0][0 0 0 0]]...]
Prediction tensor in IOU loss: [[[-0.0916902 0.887826204 -0.670153 0.0180809647][-1.30561447 -0.733022928 -0.626737714 0.740641832]]...]
Target tensor in IOU loss: [[[0 0 0 0][0 0 0 0]]...]