deepjavalibrary / deepjavalibrary/djl
org.tensorflow.exceptions.TFInvalidArgumentException: Incompatible shapes: [1,14,14,128] vs. [1,20,20,128]
- Dominant language
- Java
- Stars
- 4.9k
- Forks
- 759
- Avg merge
- 19h 26m
- Merged PRs (30d)
- 17
Description
## Description
Greetings,
I'm using a custom-made model with DJL for object detection but it's giving me an error:
Incompatible shapes: [1,14,14,128] vs. [1,20,20,128] (For MOBILENET_V2_I320)
&
Incompatible shapes: [1,14,14,128] vs. [1,16,16,128] (For MOBILENET_V2)
### Expected Behavior
It should return a JSON with the expected coordinates of the objects in the analyzed image
### Error Message
org.tensorflow.exceptions.TFInvalidArgumentException: Incompatible shapes: [1,14,14,128] vs. [1,20,20,128]
[[{{function_node __inference__wrapped_model_117326}}{{node object_detector_model/retina_net_model/fpn/add/add}}]]] with root cause
## How to Reproduce?
Use the following controller method in Java to reproduce the error:
@GetMapping("/predict_custom_object")
public static String predictCustomObject() throws IOException, ModelException, TranslateException {
Path imageFile = Paths.get("input/object_recognition/beatles.jpeg");
Image img = ImageFactory.getInstance().fromFile(imageFile);
String modelUrl = "http://download.tensorflow.org/models/object_detection/tf2/20200711/ssd_mobilenet_v2_320x320_coco17_tpu-8.tar.gz";
Criteria criteria =
Criteria.builder()
.optApplication(Application.CV.OBJECT_DETECTION)
.setTypes(Image.class, DetectedObjects.class)
.optModelUrls(modelUrl)
// saved_model.pb file is in the subfolder of the model archive file
.optModelName("saved_model")
.optTranslator(new MyTranslator())
.optEngine("TensorFlow")
.optProgress(new ProgressBar())
.build();
try (ZooModel model = criteria.loadModel();
Predictor predictor = model.newPredictor()) {
DetectedObjects detection = predictor.predict(img);
examplesService.saveBoundingBoxImage(img, detection);
return detection.toString();
}
}
public static final class MyTranslator
implements NoBatchifyTranslator {
private Map classes;
private int maxBoxes;
private float threshold;
MyTranslator() {
maxBoxes = 10;
threshold = 0.7f;
}
/** {@inheritDoc} */
@Override
public NDList processInput(TranslatorContext ctx, Image input) {
// input to tf object-detection models is a list of tensors, hence NDList
NDArray array = input.toNDArray(ctx.getNDManager(), Image.Flag.COLOR);
// optionally resize the image for faster processing
array = NDImageUtils.resize(array, 224);
// tf object-detection models expect 8 bit unsigned integer tensor
array = array.toType(DataType.FLOAT32, true);
array = array.expandDims(0); // tf object-detection models expect a 4 dimensional input
return new NDList(array);
}
/** {@inheritDoc} */
@Override
public void prepare(TranslatorContext ctx) throws IOException {
if (classes == null) {
classes = examplesService.loadSynset();
}
}
/** {@inheritDoc} */
@Override
public DetectedObjects processOutput(TranslatorContext ctx, NDList list) {
// output of tf object-detection models is a list of tensors, hence NDList in djl
// output NDArray order in the list are not guaranteed
int[] classIds = null;
float[] probabilities = null;
NDArray boundingBoxes = null;
for (NDArray array : list) {
if ("detection_boxes".equals(array.getName())) {
boundingBoxes = array.get(0);
} else if ("detection_scores".equals(array.getName())) {
probabilities = array.get(0).toFloatArray();
} else if ("detection_classes".equals(array.getName())) {
// class id is between 1 - number of classes
classIds = array.get(0).toType(DataType.INT32, true).toIntArray();
}
}
Objects.requireNonNull(classIds);
Objects.requireNonNull(probabilities);
Objects.requireNonNull(boundingBoxes);
List retNames = new ArrayList<>();
List retProbs = new ArrayList<>();
List retBB = new ArrayList<>();
// result are already sorted
for (int i = 0; i < Math.min(classIds.length, maxBoxes); ++i) {
int classId = classIds[i];
double probability = probabilities[i];
// classId starts from 1, -1 means background
if (classId > 0 && probability > threshold) {
String className = classes.getOrDefault(classId, "#" + classId);
float[] box = boundingBoxes.get(i).toFloatArray();
float yMin = box[0];
float xMin = box[1];
float yMax = box[2];
float xMax = box[3];
Rectangle rect = new Rectangle(xMin, yMin, xMax - xMin, yMax - yMin);
retNames.add(className);
retProbs.add(probability);
retBB.add(rect);
}
}
return new DetectedObjects(retNames, retProbs, retBB);
}
}
### Steps to reproduce
I used POSTMAN to call the REST endpoint
## What have you tried to solve it?
I've trained models in mediapipe_model_maker both in MOBILENET_V2_I320 and MOBILENET_V2
## Environment Info
I'm using a Spring Boot project to test the DJL on a Macbook
Contributor guide
Assessment
This issue has not been assessed yet.