tensorflow / tensorflow/models

"Input shape axis 0 must equal 1" when using EfficientDet D0 on tensorflow-serving 2.3.0

Open
#9,144 2 comments 1 reaction 1 assignee View on GitHub

@jaeyounkim is already working on this.

Since Aug 25, 2020.

models:research type:bug
Dominant language
Python
Stars
77.7k
Forks
44.8k
PR merge metrics
No merged PRs in 30d

Description

I train the model normally using model_main_tf2.py passing the pipeline_config_path using the ssd_efficientdet_d0_512x512_coco17_tpu-8.config pipeline config, which refers to the model of efficientdet d0.

It trains successfully as I used to do with faster_rcnn_resnet101 (tf 1.5).

I export the model using the script exporter_main_v2.py. It exports succesfully and it can be normally loaded by tensorflow-serving.

os.system(f'python -u exporter_main_v2.py \
            --input_type image_tensor \
            --pipeline_config_path={pipeline_file} \
            --trained_checkpoint_dir={training_dir} \
            --output_directory={serving_frozen_graph_dir}')

The error occurs when I try to inference batches from this model.
I execute the serving's grpc request with:

# tf-serving communication setup
options = [('grpc.max_receive_message_length', 100 * 4000 * 4000)]
channel = grpc.insecure_channel('{host}:{port}'.format(host='localhost', port=str(self.serving_grpc_port)), options = options)
stub = prediction_service_pb2_grpc.PredictionServiceStub(channel)
request = predict_pb2.PredictRequest()
request.model_spec.name = 'ivi-detector'
request.model_spec.signature_name = 'serving_default'
request.inputs['input_tensor'].CopyFrom(tf.make_tensor_proto(imgs_array, shape=imgs_array.shape))

res = stub.Predict(request, 100.0)

boxes = np.reshape(res.outputs['detection_boxes'].float_val, [len(imgs_array), 100,4])
classes = np.reshape(res.outputs['detection_classes'].float_val, [len(imgs_array), 100])
scores = np.reshape(res.outputs['detection_scores'].float_val, [len(imgs_array), 100])

And it returns the following error:

debug_error_string = "{"created":"@1598292999.094988994","description":"Error received from peer ipv6:[::1]:8500","file":"src/core/lib/surface/call.cc","file_line":1061,"grpc_message":"2 root error(s) found.\n  (0) Invalid argument: Input shape axis 0 must equal 1, got shape [10,1560,840,3]\n\t [[{{node Preprocessor/unstack}}]]\n\t [[StatefulPartitionedCall/StatefulPartitionedCall/Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/Reshape_7/_1463]]\n  (1) Invalid argument: Input shape axis 0 must equal 1, got shape [10,1560,840,3]\n\t [[{{node Preprocessor/unstack}}]]\n0 successful operations.\n0 derived errors ignored.","grpc_status":3}"

If I try to process using single images it works normally. I do it with:

image_expanded = np.expand_dims(imgs_array[0], axis=0)
request.inputs['input_tensor'].CopyFrom(tf.make_tensor_proto(image_expanded, shape=image_expanded.shape))
res = stub.Predict(request, 10.0)

boxes = np.reshape(res.outputs['detection_boxes'].float_val, [100,4])
classes = res.outputs['detection_classes'].float_val
scores = res.outputs['detection_scores'].float_val

Before started using efficientdet I was using faster_rcnn_resnet101_coco and it worked normally with both single images and batched images (I think that the shape=imgs_array.shape was doing the job).

Is this a bug or some configuration that I skipped when exporting the model?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.