willowtreeapps / willowtreeapps/vocable-android
Handle face tracking directly, without using ArFragment()
Nobody has claimed this yet.
- Dominant language
- Kotlin
- Stars
- 127
- Forks
- 21
- Avg merge
- 16m
- Merged PRs (30d)
- 4
Description
Currently, face tracking is handled by adding an invisible Fragment that extends ArFragment and feeds face tracking updates to a shared ViewModel observed by MainActivity.
This works, and that Fragment handles things like permissions and Google Play services usage for us automatically, however that automatic handling has some drawbacks:
- The app closes if permission is denied (we've hacked around this)
- The app can't run without Google Play services (the app should still run, just without face tracking)
- Older devices that don't support AR can't run the app at all (the app should still run, just without face tracking)
- The shared ViewModel pattern is an opportunity for misdirection
Because of these drawbacks, let's use ArSceneView directly, like ArFragment does under the hood, or like this example - https://github.com/PaulKlauser/face-tracker-demo/blob/main/app/src/main/java/com/example/facetracker/MainActivity.kt (this example is in Compose, but the SceneView idea is the same)
We should approach this incrementally, with some sort of basic feature flag in place so that we can merge in the new code without breaking the existing functionality, and then switch over once it's all merged in. No big-bang-here's-all-of-the-functionality-at-once PRs please :)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in the current ArFragment-based face-tracking flow and MainActivity, then compare it with the linked face-tracker-demo MainActivity.kt example using ArSceneView directly. Trace the shared ViewModel and define the basic feature-flag path described in the issue. Done means the new approach can be merged incrementally without breaking existing functionality, including permission denial, missing Google Play services, and unsupported devices.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- android, kotlin
- Domain
- computer-vision, mobile-dev
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100