infinitered / infinitered/react-native-mlkit
Can't detect faces in an image on iOS device
- Dominant language
- TypeScript
- Stars
- 444
- Forks
- 32
- PR merge metrics
- No merged PRs in 30d
Description
Hi there,
So I have been using this library for Android with no issues, it detects face with ease, however, on iOS device, the `faces` property is just an empty array, its not able to detect any face whatsoever.
Here's the screen file
```
//facial-scan.tsx
import React, { useRef, useState } from 'react';
import { Button, Image, View, StyleSheet, Dimensions, Text } from 'react-native';
import { Camera, useCameraDevice, useCameraPermission } from 'react-native-vision-camera';
import Svg, { Rect, Mask } from 'react-native-svg';
import { useRouter } from 'expo-router';
import { layout } from '@/styles/common';
import { Button as ButtonComponent, Header } from '@/components';
import { typography } from '@/styles/typography';
import { useIsFocused } from '@react-navigation/native';
import { useFacesInPhoto, useFaceDetection } from '@infinitered/react-native-mlkit-face-detection';
const { width, height } = Dimensions.get('window');
const horizontalPadding = 20;
const maskMarginTop = 30;
const frameWidth = width - horizontalPadding * 2;
const frameHeight = frameWidth;
const frameX = (width - width * 0.9) / 2;
const frameY = maskMarginTop;
export default function FaceRecognitionScreen() {
const model = useFaceDetection();
const device = useCameraDevice('front');
const router = useRouter();
const [photoUri, setPhotoUri] = useState(null);
const { hasPermission, requestPermission } = useCameraPermission();
// This hook returns `true` if the screen is focused, `false` otherwise
const isFocused = useIsFocused();
// const appState = useAppState()
const isActive = isFocused;
const cameraRef = useRef(null);
const { faces, error } = useFacesInPhoto(photoUri ?? undefined);
const takePicture = async () => {
if (!cameraRef.current) return;
try {
const photo = await cameraRef.current?.takePhoto({ flash: 'off' });
try {
const result = await model.detectFaces(`file://${photo.path}`);
// Comment below line prior to submitting new release APK
// router.replace('/(auth)/review-info');
console.log('result: ', result)
// Uncomment below line prior during development with physical device
if (!result || !faces.length) {
// Handle undefined result case
alert('No faces detected!');
return;
}
else if(faces.length === 1) {
router.navigate('/(auth)/review-info');
}
alert(`Faces detected!, ${faces.length}, ${JSON.stringify(faces)}`);
} catch (error) {
// Handle error case
console.error('Face detection failed:', error);
}
} catch (err) {
console.error('Error taking photo:', err);
}
};
return (
{
router.back();
}} />
{/* Camera */}
{device && (
{!hasPermission ? (
We need your permission to show the camera
) : }
)}
{/* Masked Overlay */}
{/* Text inside frame */}
Face Recognition
Try to keep your face within the frame
);
}
const styles = StyleSheet.create({
container: {
flex: 1,
borderWidth: 3,
borderColor: 'red'
},
backButton: {},
title: {
fontSize: 18,
fontWeight: 'bold',
color: '#00C2CB',
},
subtitle: {
fontSize: 14,
color: '#fff',
marginTop: 5,
borderWidth: 2,
borderColor: 'red'
},
submitButton: {
position: 'absolute',
bottom: 40,
alignSelf: 'center',
backgroundColor: '#00C2CB',
// borderRadius: 999,
paddingVertical: 12,
paddingHorizontal: 40,
},
submitText: {
color: '#fff',
fontSize: 16,
fontWeight: 'bold',
},
textInFrameContainer: {
position: 'absolute',
alignItems: 'center',
justifyContent: 'space-evenly',
top: frameHeight * 0.65,
left: frameX - 20,
width: frameWidth,
height: 150
},
camera: {
width: '99.9%',
height: height * 0.75,
overflow: 'hidden',
borderColor: 'green',
borderWidth: 2
},
cameraContainer: {
marginTop: 20,
overflow: 'hidden',
flex: 1,
},
roundedCameraWrapper: {
width: '99.9%',
height: height * 0.75,
borderRadius: 10, // whatever radius you want
overflow: 'hidden',
alignSelf: 'center', // optional, if needed
borderWidth: 2,
borderColor: 'green', // optional
},
permissionPromptText: {
...typography.body,
color: 'white',
marginBottom: 10
},
});
```
and here is output from
```
await model.detectFaces(`file://${photo.path}`):
```
`{"faces": [], "imagePath": "file://file///private/var/mobile/Containers/Data/Application/AD8AF7CF-043A-4125-970F-5ECB662C9016/tmp/9B9AEF52-9647-4259-9A3F-AA3BAFC6EB6B.jpg"}`
Contributor guide
Research direction
Start with the facial-scan.tsx example and the useFaceDetection().detectFaces call, then inspect the iOS handling for the file URL shown in the reported output. Reproduce the same photo on iOS and Android and compare the returned faces arrays and image paths. Done means the iOS case is explained and face detection works consistently or the limitation is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ios, react-native, typescript
- Domain
- mobile
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100