aws-amplify / aws-amplify/amplify-ui
Infrared Camera Incorrectly Selected for Liveness Detection on Pixel 4
- Dominant language
- TypeScript
- Stars
- 1.1k
- Forks
- 347
- Avg merge
- 18h 29m
- Merged PRs (30d)
- 9
Description
### Before creating a new issue, please confirm:
- [X] I have [searched for duplicate or closed issues](https://github.com/aws-amplify/amplify-ui/issues?q=is%3Aissue+) and [discussions](https://github.com/aws-amplify/amplify-ui/discussions).
- [X] I have tried disabling all browser extensions or using a different browser
- [X] I have tried deleting the node_modules folder and reinstalling my dependencies
- [X] I have read the guide for [submitting bug reports](https://github.com/aws-amplify/amplify-ui/blob/main/CONTRIBUTING.md#bug-reports).
### On which framework/platform are you having an issue?
React
### Which UI component?
Liveness
### How is your app built?
Create React App
### What browsers are you seeing the problem on?
Android (React Native)
### Which region are you seeing the problem in?
_No response_
### Please describe your bug.
The FaceLivenessDetector component incorrectly selects the infrared camera (used for face unlock) instead of the regular selfie camera when performing liveness validation on Pixel 4 devices. This causes the liveness detection to fail as it attempts to use the infrared camera feed instead of the normal camera feed.
Environment:
- Device: Pixel 4
- Packages:
@aws-amplify/ui-react: 6.1.10
@aws-amplify/ui-react-liveness: 3.0.23
### What's the expected behaviour?
The regular front-facing (selfie) camera should be activated for liveness detection.
### Help us reproduce the bug!
Steps to Reproduce
- Initialize FaceLivenessDetector component
- Attempt to perform liveness validation
- Observe that the infrared camera is activated instead of the normal selfie camera
### Code Snippet
```javascript
return (
{!sessionId ? (
) : (
{
console.error(error);
}}
displayText={displayText}
onUserCancel={() => sendMessageToNative({ status: "CLOSED" })}
/>
)}
);
```
### Console log output
_No response_
### Additional information and screenshots
The infrared camera (used for face unlock) is being detected as the first available camera on getUserMedia call, causing the liveness detection to fail.
The issue occurs in:
packages/react-liveness/src/components/FaceLivenessDetector/service/machine/machine.ts
in the checkVirtualCameraAndGetStream function.
Here is my suggestion:
Filter out infrared cameras from the device selection process by:
1. Adding infrared camera detection
2. Filtering out infrared cameras from real video devices
3. Validating existing devices against infrared camera check
Here are some references that made me think this can be solved like above:
- [Pixel 4 XL Infrared Sensor via getUserMedia](https://paul.kinlan.me/pixel-4xl-infrared-sensor-via-getusermedia/)
- [Chromium Review - IR Camera Changes](https://chromium-review.googlesource.com/c/chromium/src/+/1960152)
Here is a screenshot
Contributor guide
Research direction
Start in packages/react-liveness/src/components/FaceLivenessDetector/service/machine/machine.ts, focusing on checkVirtualCameraAndGetStream and how getUserMedia devices are selected. Reproduce on a Pixel 4 running Android and compare the available camera devices with the selected stream. Done means liveness uses the regular front-facing camera rather than the infrared camera.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- react, react-native, typescript
- Domain
- frontend, mobile-dev
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100