willowtreeapps / willowtreeapps/vocable-android

Handle face tracking directly, without using ArFragment()

Open
#456 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Kotlin
Stars
127
Forks
21
Avg merge
16m
Merged PRs (30d)
4

Description

Currently, face tracking is handled by adding an invisible Fragment that extends ArFragment and feeds face tracking updates to a shared ViewModel observed by MainActivity.

This works, and that Fragment handles things like permissions and Google Play services usage for us automatically, however that automatic handling has some drawbacks:

  • The app closes if permission is denied (we've hacked around this)
  • The app can't run without Google Play services (the app should still run, just without face tracking)
  • Older devices that don't support AR can't run the app at all (the app should still run, just without face tracking)
  • The shared ViewModel pattern is an opportunity for misdirection

Because of these drawbacks, let's use ArSceneView directly, like ArFragment does under the hood, or like this example - https://github.com/PaulKlauser/face-tracker-demo/blob/main/app/src/main/java/com/example/facetracker/MainActivity.kt (this example is in Compose, but the SceneView idea is the same)

We should approach this incrementally, with some sort of basic feature flag in place so that we can merge in the new code without breaking the existing functionality, and then switch over once it's all merged in. No big-bang-here's-all-of-the-functionality-at-once PRs please :)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in the current ArFragment-based face-tracking flow and MainActivity, then compare it with the linked face-tracker-demo MainActivity.kt example using ArSceneView directly. Trace the shared ViewModel and define the basic feature-flag path described in the issue. Done means the new approach can be merged incrementally without breaking existing functionality, including permission denial, missing Google Play services, and unsupported devices.

Written by the indexing model from the issue text.

Assessment

Tech stack
android, kotlin
Domain
computer-vision, mobile-dev
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.