Tencent / Tencent/ncnn

ncnn android with 1 channel y model

Open
#4,366 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
23.8k
Forks
4.5k
Avg merge
2d 20h
Merged PRs (30d)
37

Description

Hello,
I am experimenting with ncnn in my android app to replace tensorflow lite.
My models most of time use Y channel (1C) instead of RGB (3channel), reason for it is speed.
When I am running my 1 channel models on my pc application code (QT application) using ncnn, results are fine.
But when I run that model on android 11 it show broken pixels. (please refer to image below).

In android, I have a shader code which converts Bitmap from BGRA (little endia) to Y channel and which I am feeding to ncnn model like this

void *data = (void *)env->GetDirectBufferAddress(image_buffer);
ncnn::Mat in = ncnn::Mat(w, h, c, data, (size_t)4u);
.
.
ncnn::Mat out;
ret = ex.extract(oidxs[0], out);

and then I am returning results like this
jobject obj = env->NewDirectByteBuffer(out.data, (out.wout.hout.c*out.elemsize));

and after results are return my code is converting Y back to BGRA to draw on app.

Screenshot_20221123-094244_GDFSRPlayer~2

you can see whole white background is looking like a net with holes.

Note: I am using same shader and same model with tensor-flow lite on same application and results are fine.
am I missing something?

P.s. I tried 3 channel model also when I feed data using
ncnn::Mat in = ncnn::Mat(w, h, c, data, (size_t)4u);
instead of
ncnn::Mat::from_android_bitmap
it shows same results as above.

Please guide

Thank you in advance

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Android path using ncnn::Mat(w, h, c, data, (size_t)4u), compare it with ncnn::Mat::from_android_bitmap, and inspect the buffer layout and output conversion to BGRA. Reproduce the mismatch against the working PC and TensorFlow Lite paths; done means the Android output no longer shows broken pixels for the same model and input.

Written by the indexing model from the issue text.

Assessment

Tech stack
android, cpp, tensorflow
Domain
machine-learning, mobile-dev
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.