onnx / onnx/models

Equivalent for convert np.array input from PILImage to Tensor input in Java

Open
#245 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
9.8k
Forks
1.6k
PR merge metrics
No merged PRs in 30d

Description

I have successfully converted the emotion_ferplux.onnx model to a Tensorflow protobuf file and am trying to execute the model in java. In the python examples we have a code snippet:

input_shape = (1, 1, 64, 64)
img = Image.open(image_path)
img = img.resize((64, 64), Image.ANTIALIAS)
img_data = np.array(img) //Needs a dtype=float32
img_data = np.resize(img_data, input_shape)

to create the [1,1,64,64] input for the onnx model. From what I've been able to find over on the Java side in Tensorflow, there's something like this code snippet mentioned in the link here

[https://github.com/tensorflow/tensorflow/blob/master/tensorflow/java/src/main/java/org/tensorflow/examples/LabelImage.java]

      final Output<String> input = b.constant("input", imageBytes);
      final Output<Float> output =
          b.div(
              b.sub(
                  b.resizeBilinear(
                      b.expandDims(
                          b.cast(b.decodeJpeg(input, 3), Float.class),
                          b.constant("make_batch", 0)),
                      b.constant("size", new int[] {H, W})),
                  b.constant("mean", mean)),
              b.constant("scale", scale));

However this produces a [?][224][224][3]

I've been fiddling around to change the channel to use a value of 1 and also the dimensions to match the 64x64 input from the onnx model, however, when I inspect the actual values I'll be sending into the Tensorflow pb model over in java, they are very different.

I'm led to believe I have to create the input similar to how PILImage and Numpy are representing the image. I've also stumbled across some code from the org.bytedeco libraries, but haven't been able to piece together the equivalent.

Does anyone have any pointers on how to go about this?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Python preprocessing snippet in the issue and TensorFlow's Java example, LabelImage.java. Compare how each path loads, resizes, channels, reshapes, and types the image input. Done means documenting or demonstrating an agreed Java equivalent that produces the model's [1,1,64,64] float32 input.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, tensorflow
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.