[object detection] support for per-example partial labels

Prerequisites

Please answer the following question for yourself before submitting an issue.

• I checked to make sure that this feature has not been requested already.

1. The entire URL of the file you are using

The following branch contains my changes to enable support for this feature:
https://github.com/staubda/models/tree/object_detection_partial_data

Files affected include:

• https://github.com/tensorflow/models/blob/master/research/object_detection/core/losses.py
• https://github.com/tensorflow/models/blob/master/research/object_detection/data_decoders/tf_example_decoder.py
• https://github.com/tensorflow/models/blob/master/research/object_detection/inputs.py
• https://github.com/tensorflow/models/blob/master/research/object_detection/meta_architectures/ssd_meta_arch.py

2. Describe the feature you request

I'd like to contribute a PR adding per-example support for partial labels to the object detection repo. Each example image should be (optionally) bundled with a list of classes for which labels were annotated, which may include any subset of the complete taxonomy supported by the model. During training each example should only contribute loss for the classes it supports.

This is particularly relevant when combining datasets with images from the same domain labeled with partially overlapping or non-overlapping taxonomies, with the goal of training a model supporting a union of the taxonomies. E.g. dataset1 is annotated with labels for {car, motorcycle, truck}, dataset2 is annotated with labels for {motorcycle, pedestrian, bicycle}, and the detection model should support {car, motorcycle, truck, pedestrian, bicycle}. In order to train the best model, predictions of, e.g., bicycle, on an example from dataset1 should not be penalized since a bicycle may actually be present but will never be labeled.

3. Additional context

Primary changes are to

• Loss function
• Detection meta-arch
• TFRecord deserialization

I've tested the feature on a synthetic dataset mimicking the above scenario (a subset of COCO images with labels partially removed) and recorded the results in this repo https://github.com/staubda/partial-data/tree/master/object_detection/trained_models. Specifically, runs 4, 5, and 6 show results on an identical test and train sets, but with:

• run 4: partial labels, no per-example support for specifying and masking unsupported classes when computing loss - mAP@0.5 = 0.196
• run 5: complete labels - mAP@0.5 = .251
• run 6: partial labels, includes per-example support for specifying and masking unsupported classes when computing loss - mAP@0.5 = 0.201

The repo includes all notebooks and scripts necessary for recreating datasets and training runs.

4. Are you willing to contribute it? (Yes or No)

Yes, PR is ready to go for SSD meta-arch with weighted sigmoid classification loss.