deepjavalibrary / deepjavalibrary/djl

Error issue using BatchPredict API

Open
#3,401 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
4.9k
Forks
759
Avg merge
16h 15m
Merged PRs (30d)
15

Description

Hi,
I use BatchPredict API for batch prediction, and when the input data shape is NDList size: 1,
0 : (2, 64) int32 or NDList size: 2 0 : (64) int32,1 : (64) int32, it will appear RuntimeError: The size of tensor a (64) must match the size of tensor b (43) at non-singleton dimension 2,
but shape is NDList size: 1
0 : (64) int32, it's ok.
So I would like to ask if batchPredict can perform batch prediction?If possible, what format should be entered for the shape?

Thank you very much!

```
Criteria criteria =
Criteria.builder()
.setTypes(NDList.class, Float.class)
.optModelPath(Paths.get(modelPath))
.optTranslator(new MyTranslator())
.optEngine("PyTorch")
.build();
ZooModel model = ModelZoo.loadModel(criteria);
predictor = model.newPredictor();

public static class MyTranslator implements Translator {
@Override
public Batchifier getBatchifier() {
return Batchifier.STACK;
}
@Override
public NDList processInput(TranslatorContext translatorContext, NDList inputs) { return inputs; }
@Override
public Float processOutput(TranslatorContext ctx, NDList outputs) {
return outputs.get(0).getFloat();
}
}

predictor.batchPredict(Collections.singletonList(inputs));
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.