benaclejames / benaclejames/VRCFaceTracking
[Feature Request] better support of mix-and-matching tracking sources from multiple modules
- Dominant language
- C#
- Stars
- 907
- Forks
- 135
- Avg merge
- 11h 28m
- Merged PRs (30d)
- 7
Description
A recurring issue with VRCFT has been trying to mix and match tracking sources across modules. The most recent influx of questions regarding this is people wanting to use Babble face trackers on a Quest Pro. When the Virtual Desktop or Steam Link modules load, they rightly claim both eye and face tracking. If the Babble module loads after this, it can no longer claim face tracking, so users are stuck with the face tracking data from the Quest Pro itself.
A quick and dirty fix to this could be to automatically load eye and face support for all modules, then giving users the option to choose which module they want to use for eyes, and which module they want to use for face.
Replace the "Tracking Enabled" checkbox next to each module on the main VRCFT window with two separate "track eyes" and "track face" checkboxes. Ideally only allow one eye and one face to be checked. The checkboxes would then just need to update `AvailableSandboxModules[moduleIndex].ModuleInformation.UsingEye/UsingExpression` in `VRCFaceTracking.Core/Library/UnifiedLibManager.cs`.
A more advanced solution to this would be allowing users to pick and choose individual expressions from different tracking modules. These expression sets could then be saved as "presets" that users can switch between. The benefit to this is that users may prefer some expressions from one module source, while they want others from another. They might like how the Quest Pro cameras handle frowning, but want the tongue expressions from Babble.
This would also allow a wider range of modules to exist. For example, azmidi's VD module fork contains a small program that reads the VRC screen brightness and fakes pupil dilation based off of it. This could be released as its own individual module that users install, and then they just need to select pupil dilation from that module.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with VRCFaceTracking.Core/Library/UnifiedLibManager.cs and the main VRCFT window where the module Tracking Enabled checkbox is rendered. Compare the proposed separate eye and face selections with the existing UsingEye and UsingExpression fields. Done should allow independent module selection for eye and face tracking, while the expression-level mixing and preset ideas require a design decision.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- desktop
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100