tensorflow / tensorflow/tflite-support

Android: RGB bitmap from YUV image

Open
#541 13 comments 1 reaction 1 assignee View on GitHub

@lu-wang-g is already working on this.

Since Jun 10, 2021.

Dominant language
C++
Stars
441
Forks
146
PR merge metrics
No merged PRs in 30d

Description

v0.2.0 introduced support for YUV android.media.Image as part of the tensorflow-lite-task-vision artifact. Seems like it uses libyuv under the hood. As far as I understand TensorImage delegates to MediaImageContainer in this case. At the moment of writing it doesn’t support the getBitmap method, throwing an exception.

We have such use case:

  1. take a YUV image from camera;
  2. convert and rotate the image to a RGB Bitmap via RenderScript;
  3. pass the Bitmap to TF;
  4. save the Bitmap as a JPEG file alongside detections done by TF.

We don’t grab an RGB bitmap from camera directly since it takes much more time than grabbing a YUV image and processing it via RenderScript. However, the Android 12 tooling deprecates RenderScript.

It would be great if the MediaImageContainer.getBitmap method returned a valid Bitmap. I imagine in this scenario our use case might avoid using RenderScript. As far as I understood, the byte buffer for rotated and converted image exists during the TF processing already, it just needs to be shifted to a Bitmap on demand.

If this is not what TF wants to support — can I ask for pointers on best practices regarding something like this? Meaning reading the camera frame, processing it via TF and then saving the frame.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.