tensorflow / tensorflow/models

Unable to do "out-of-the-box" inference using the CenterNet Hourglass104 Keypoints Model - Object detection

Open
#9,647 1 comment 0 reactions 1 assignee View on GitHub

@pkulzc is already working on this.

Since Feb 9, 2021.

models:research stat:awaiting model gardener type:support
Dominant language
Python
Stars
77.7k
Forks
44.8k
PR merge metrics
No merged PRs in 30d

Description

  • [ X] I am using the latest TensorFlow Model Garden release and TensorFlow 2.
  • [ X] I am reporting the issue to the correct repository. (Model Garden official or research directory)
  • [ X] I checked to make sure that this issue has not already been filed.

The entire URL of the file you are using

https://github.com/tensorflow/models/blob/master/research/object_detection/colab_tutorials/inference_tf2_colab.ipynb

Describe the bug

On request opening a fresh issues reference to #9507

Steps to reproduce

Changed models dictionary to centernet_hourglass104_512x512_kpts_coco17_tpu-32 to download and extract the correct model.

Pick up the config from this location for the above model
pipeline_config = .../research/object_detection/configs/tf2/centernet_hourglass104_512x512_kpts_coco17_tpu-32.config

Changes made in the above config file -
line 44 --> keypoint_label_map_path: ".../research/object_detection/data/face_person_with_keypoints_label_map.pbtxt"
line 60 --> keypoint_class_name: "Person"
line 241 --> (in eval_cong) class_label: "Person" as suggested in #9507

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 16.04
  • TensorFlow version (use command below): v2.2.0-rc4-8-g2b96f3662b 2.2.0
  • Python version: 3.6.10

Expected behavior

Run the test image successfully and return predictions.

Code from colab

#Testing on image from test_images folder
image_dir = '.../research/object_detection/test_images/'
image_path = os.path.join(image_dir, 'image2.jpg')
image_np = load_image_into_numpy_array(image_path)

input_tensor = tf.convert_to_tensor(
    np.expand_dims(image_np, 0), dtype=tf.float32)
detections, predictions_dict, shapes = detect_fn(input_tensor)
Error:
TypeError                                 Traceback (most recent call last)
<ipython-input-26-53d3633f8f72> in <module>
     13 input_tensor = tf.convert_to_tensor(
     14     np.expand_dims(image_np, 0), dtype=tf.float32)
---> 15 detections, predictions_dict, shapes = detect_fn(input_tensor)
     16 
     17 label_id_offset = 1

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py in __call__(self, *args, **kwds)
    578         xla_context.Exit()
    579     else:
--> 580       result = self._call(*args, **kwds)
    581 
    582     if tracing_count == self._get_tracing_count():

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py in _call(self, *args, **kwds)
    625       # This is the first call of __call__, so we have to initialize.
    626       initializers = []
--> 627       self._initialize(args, kwds, add_initializers_to=initializers)
    628     finally:
    629       # At this point we know that the initialization is complete (or less

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py in _initialize(self, args, kwds, add_initializers_to)
    504     self._concrete_stateful_fn = (
    505         self._stateful_fn._get_concrete_function_internal_garbage_collected(  # pylint: disable=protected-access
--> 506             *args, **kwds))
    507 
    508     def invalid_creator_scope(*unused_args, **unused_kwds):

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow/python/eager/function.py in _get_concrete_function_internal_garbage_collected(self, *args, **kwargs)
   2444       args, kwargs = None, None
   2445     with self._lock:
-> 2446       graph_function, _, _ = self._maybe_define_function(args, kwargs)
   2447     return graph_function
   2448 

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow/python/eager/function.py in _maybe_define_function(self, args, kwargs)
   2775 
   2776       self._function_cache.missed.add(call_context_key)
-> 2777       graph_function = self._create_graph_function(args, kwargs)
   2778       self._function_cache.primary[cache_key] = graph_function
   2779       return graph_function, args, kwargs

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow/python/eager/function.py in _create_graph_function(self, args, kwargs, override_flat_arg_shapes)
   2665             arg_names=arg_names,
   2666             override_flat_arg_shapes=override_flat_arg_shapes,
-> 2667             capture_by_value=self._capture_by_value),
   2668         self._function_attributes,
   2669         # Tell the ConcreteFunction to clean up its graph once it goes out of

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow/python/framework/func_graph.py in func_graph_from_py_func(name, python_func, args, kwargs, signature, func_graph, autograph, autograph_options, add_control_dependencies, arg_names, op_return_value, collections, capture_by_value, override_flat_arg_shapes)
    979         _, original_func = tf_decorator.unwrap(python_func)
    980 
--> 981       func_outputs = python_func(*func_args, **func_kwargs)
    982 
    983       # invariant: `func_outputs` contains only Tensors, CompositeTensors,

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py in wrapped_fn(*args, **kwds)
    439         # __wrapped__ allows AutoGraph to swap in a converted function. We give
    440         # the function a weak reference to itself to avoid a reference cycle.
--> 441         return weak_wrapped_fn().__wrapped__(*args, **kwds)
    442     weak_wrapped_fn = weakref.ref(wrapped_fn)
    443 

~/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
    966           except Exception as e:  # pylint:disable=broad-except
    967             if hasattr(e, "ag_error_metadata"):
--> 968               raise e.ag_error_metadata.to_exception(e)
    969             else:
    970               raise

TypeError: in user code:

    <ipython-input-17-04098793256e>:27 detect_fn  *
        detections = model.postprocess(prediction_dict, shapes)
    /home/ubuntu/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/object_detection/meta_architectures/center_net_meta_arch.py:2945 postprocess  *
        keypoints, keypoint_scores = (
    /home/ubuntu/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/object_detection/meta_architectures/center_net_meta_arch.py:1086 convert_strided_predictions_to_normalized_keypoints  *
        keypoint_coords_normalized = tf.map_fn(

    TypeError: map_fn() got an unexpected keyword argument 'fn_output_signature'

Let me know if something is missing.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.