googlesamples / googlesamples/mlkit
[Bug report] Barcode AutoZoom not working with coordinate matrix in InputImage
- Dominant language
- Kotlin
- Stars
- 4.3k
- Forks
- 3.1k
- PR merge metrics
- No merged PRs in 30d
Description
**Describe the bug**
The auto-zoom feature of the barcode scanner does not work when using a `InputImage` that has a coordinate matrix attached.
In my code I use the matrix to automatically map the ml-kit barcode coordinates to my preview views coordinate system.
**To Reproduce**
1. Create a barcode scanner with configured auto-zoom
2. In the `ImageAnalysis` create the `InputImage` using a `Matrix` and pass the image to the `process` function
4. No auto-zoom will happen
The code to create the Matrix is the same as used in the [`MlKitAnalyzer`](https://developer.android.com/media/camera/camerax/mlkitanalyzer) and does successfully map the coordinates.
When not using a custom matrix I can see the following logcat output and the camera is zoomed in:
```
I Process PredictedArea: [0,58, 0,00, 0,72, 0,66, 0,00], frameIndex = 101
I Process PredictedArea: [0,59, 0,01, 0,75, 0,36, 0,00], frameIndex = 102
I Process PredictedArea: [0,52, 0,05, 0,74, 0,29, 0,00], frameIndex = 105
I Process PredictedArea: [0,52, 0,06, 0,74, 0,28, 0,00], frameIndex = 106
I Process PredictedArea: [0,52, 0,01, 0,75, 0,27, 0,00], frameIndex = 107
I Process PredictedArea: [0,48, 0,01, 0,74, 0,27, 0,00], frameIndex = 108
I Going to set zoom = 1.9689503
```
With my custom matrix the following is logged and no zooming is happening (Camera is about the same distance from the code as in the first example):
```
I Process PredictedArea: [0,60, 0,54, 0,69, 0,93, 0,00], frameIndex = 1
I Process PredictedArea: [0,57, 0,52, 0,69, 0,93, 0,00], frameIndex = 2
I Process PredictedArea: [0,58, 0,60, 0,69, 0,93, 0,00], frameIndex = 3
I Process PredictedArea: [0,59, 0,64, 0,70, 0,93, 0,00], frameIndex = 4
I Process PredictedArea: [0,58, 0,61, 0,70, 0,97, 0,00], frameIndex = 5
I Process PredictedArea: [0,58, 0,59, 0,70, 0,97, 0,00], frameIndex = 6
I Auto zoom to 1.0 is filtered by threshold
```
I suspect that this might be because when mapping the coordinates using the matrix, the size of the codes is increased and thus the AutoZoom algorithm thinks that a zoom is not needed.
To solve that problem, the AutoZoom should calculate the distance based on the unmodified bounding box of the barcode instead of with the mapped coordinates.
**Expected behavior**
The zoom ratio should be updated to zoom in on far away codes.
**SDK Info:**
- com.google.android.gms:play-services-mlkit-barcode-scanning:18.3.0
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.