benaclejames / benaclejames/VRCFaceTracking

[Feature Request] better support of mix-and-matching tracking sources from multiple modules

Open
#366 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C#
Stars
907
Forks
135
Avg merge
11h 28m
Merged PRs (30d)
7

Description

A recurring issue with VRCFT has been trying to mix and match tracking sources across modules. The most recent influx of questions regarding this is people wanting to use Babble face trackers on a Quest Pro. When the Virtual Desktop or Steam Link modules load, they rightly claim both eye and face tracking. If the Babble module loads after this, it can no longer claim face tracking, so users are stuck with the face tracking data from the Quest Pro itself.

A quick and dirty fix to this could be to automatically load eye and face support for all modules, then giving users the option to choose which module they want to use for eyes, and which module they want to use for face.
Replace the "Tracking Enabled" checkbox next to each module on the main VRCFT window with two separate "track eyes" and "track face" checkboxes. Ideally only allow one eye and one face to be checked. The checkboxes would then just need to update `AvailableSandboxModules[moduleIndex].ModuleInformation.UsingEye/UsingExpression` in `VRCFaceTracking.Core/Library/UnifiedLibManager.cs`.

A more advanced solution to this would be allowing users to pick and choose individual expressions from different tracking modules. These expression sets could then be saved as "presets" that users can switch between. The benefit to this is that users may prefer some expressions from one module source, while they want others from another. They might like how the Quest Pro cameras handle frowning, but want the tongue expressions from Babble.
This would also allow a wider range of modules to exist. For example, azmidi's VD module fork contains a small program that reads the VRC screen brightness and fakes pupil dilation based off of it. This could be released as its own individual module that users install, and then they just need to select pupil dilation from that module.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with VRCFaceTracking.Core/Library/UnifiedLibManager.cs and the main VRCFT window where the module Tracking Enabled checkbox is rendered. Compare the proposed separate eye and face selections with the existing UsingEye and UsingExpression fields. Done should allow independent module selection for eye and face tracking, while the expression-level mixing and preset ideas require a design decision.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
desktop
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.