tensorflow / tensorflow/models

[object_detection] Adding a handy Colab Notebook for training a custom pets detector model using the latest API

Open
#8,836 0 comments 5 reactions 3 assignees View on GitHub

@pkulzc is already working on this.

Since Jul 13, 2020.

models:research:odapi type:support
Dominant language
Python
Stars
77.7k
Forks
44.8k
PR merge metrics
No merged PRs in 30d

Description

Prerequisites

Please answer the following question for yourself before submitting an issue.

  • I checked to make sure that this issue has not been filed already.

I prepared a comprehensive Colab Notebook so that the onboarding experience becomes as seamless as possible. The Colab Notebook mere acts as a base VM to:

  • Prepare the data
  • Copy the data over to a GCS bucket
  • Launch training and evaluation jobs (using Cloud TPUs and GPUs respectively)
  • Monitor results with TensorBoard within the Colab Notebook itself

I have used the following references to prepare this notebook:

In this document it is suggested to use 3.6 as the Python version which is actually no longer supported by AI Platform and it will cause the following error:

ERROR: (gcloud.ai-platform.jobs.submit.training) INVALID_ARGUMENT: Field: python_version Error: The specified Python version '3.6' is not supported.
- '@type': type.googleapis.com/google.rpc.BadRequest
  fieldViolations:
  - description: The specified Python version '3.6' is not supported.
    field: python_version

The fix is to change it to 3.7 and everything will be golden. I hope this Colab Notebook is going to be useful. I plan to author more notebooks showing more involved data preparation processes & popping them up on a GPU for training and that might be even more helpful for the community.

Question about export the TF 2 compatible models

I could not export the custom trained models because it appears to me that it is not documented yet anywhere. Please correct me if I am wrong. I would specifically be interested in exporting the checkpoints as a SavedModel file and then converting it to a TF Lite model.

Update: I was able to export the checkpoints after referring https://github.com/tensorflow/models/issues/8841:

python object_detection/exporter_main_v2.py \
    --input_type encoded_image_string_tensor \
    --trained_checkpoint_dir gs://$YOUR_GCS_BUCKET/$MODEL_DIR \
    --output_directory `pwd` \
    --pipeline_config_path gs://$YOUR_GCS_BUCKET/data/$PIPELINE_CONFIG_PATH

I was able to run inference with the model checkpoints as well. Here's a separate notebook that shows the process. I think it might useful for folks that are just interested in detection models only.

Question: What is the preferred format for running inference? Checkpoints or SavedModel? I find SavedModel to be more convenient with the respect of the TensorFlow ecosystem.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.