deepjavalibrary / deepjavalibrary/djl

org.tensorflow.exceptions.TFInvalidArgumentException: Incompatible shapes: [1,14,14,128] vs. [1,20,20,128]

Open
#2,811 3 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Java
Stars
4.9k
Forks
759
Avg merge
19h 26m
Merged PRs (30d)
17

Description

## Description

Greetings,
I'm using a custom-made model with DJL for object detection but it's giving me an error:
Incompatible shapes: [1,14,14,128] vs. [1,20,20,128] (For MOBILENET_V2_I320)
&
Incompatible shapes: [1,14,14,128] vs. [1,16,16,128] (For MOBILENET_V2)

### Expected Behavior
It should return a JSON with the expected coordinates of the objects in the analyzed image

### Error Message
org.tensorflow.exceptions.TFInvalidArgumentException: Incompatible shapes: [1,14,14,128] vs. [1,20,20,128]
[[{{function_node __inference__wrapped_model_117326}}{{node object_detector_model/retina_net_model/fpn/add/add}}]]] with root cause

## How to Reproduce?

Use the following controller method in Java to reproduce the error:

@GetMapping("/predict_custom_object")
public static String predictCustomObject() throws IOException, ModelException, TranslateException {
Path imageFile = Paths.get("input/object_recognition/beatles.jpeg");
Image img = ImageFactory.getInstance().fromFile(imageFile);

String modelUrl = "http://download.tensorflow.org/models/object_detection/tf2/20200711/ssd_mobilenet_v2_320x320_coco17_tpu-8.tar.gz";

Criteria criteria =
Criteria.builder()
.optApplication(Application.CV.OBJECT_DETECTION)
.setTypes(Image.class, DetectedObjects.class)
.optModelUrls(modelUrl)
// saved_model.pb file is in the subfolder of the model archive file
.optModelName("saved_model")
.optTranslator(new MyTranslator())
.optEngine("TensorFlow")
.optProgress(new ProgressBar())
.build();

try (ZooModel model = criteria.loadModel();
Predictor predictor = model.newPredictor()) {
DetectedObjects detection = predictor.predict(img);
examplesService.saveBoundingBoxImage(img, detection);
return detection.toString();
}
}

public static final class MyTranslator
implements NoBatchifyTranslator {

private Map classes;
private int maxBoxes;
private float threshold;

MyTranslator() {
maxBoxes = 10;
threshold = 0.7f;
}

/** {@inheritDoc} */
@Override
public NDList processInput(TranslatorContext ctx, Image input) {
// input to tf object-detection models is a list of tensors, hence NDList
NDArray array = input.toNDArray(ctx.getNDManager(), Image.Flag.COLOR);
// optionally resize the image for faster processing
array = NDImageUtils.resize(array, 224);
// tf object-detection models expect 8 bit unsigned integer tensor
array = array.toType(DataType.FLOAT32, true);
array = array.expandDims(0); // tf object-detection models expect a 4 dimensional input
return new NDList(array);
}

/** {@inheritDoc} */
@Override
public void prepare(TranslatorContext ctx) throws IOException {
if (classes == null) {
classes = examplesService.loadSynset();
}
}

/** {@inheritDoc} */
@Override
public DetectedObjects processOutput(TranslatorContext ctx, NDList list) {
// output of tf object-detection models is a list of tensors, hence NDList in djl
// output NDArray order in the list are not guaranteed

int[] classIds = null;
float[] probabilities = null;
NDArray boundingBoxes = null;
for (NDArray array : list) {
if ("detection_boxes".equals(array.getName())) {
boundingBoxes = array.get(0);
} else if ("detection_scores".equals(array.getName())) {
probabilities = array.get(0).toFloatArray();
} else if ("detection_classes".equals(array.getName())) {
// class id is between 1 - number of classes
classIds = array.get(0).toType(DataType.INT32, true).toIntArray();
}
}
Objects.requireNonNull(classIds);
Objects.requireNonNull(probabilities);
Objects.requireNonNull(boundingBoxes);

List retNames = new ArrayList<>();
List retProbs = new ArrayList<>();
List retBB = new ArrayList<>();

// result are already sorted
for (int i = 0; i < Math.min(classIds.length, maxBoxes); ++i) {
int classId = classIds[i];
double probability = probabilities[i];
// classId starts from 1, -1 means background
if (classId > 0 && probability > threshold) {
String className = classes.getOrDefault(classId, "#" + classId);
float[] box = boundingBoxes.get(i).toFloatArray();
float yMin = box[0];
float xMin = box[1];
float yMax = box[2];
float xMax = box[3];
Rectangle rect = new Rectangle(xMin, yMin, xMax - xMin, yMax - yMin);
retNames.add(className);
retProbs.add(probability);
retBB.add(rect);
}
}

return new DetectedObjects(retNames, retProbs, retBB);
}
}

### Steps to reproduce

I used POSTMAN to call the REST endpoint

## What have you tried to solve it?

I've trained models in mediapipe_model_maker both in MOBILENET_V2_I320 and MOBILENET_V2

## Environment Info

I'm using a Spring Boot project to test the DJL on a Macbook

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.