benaclejames / benaclejames/VRCFaceTracking

[Feature Request] Customize tracked blendshapes for combined tracking modules

Open
#236 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C#
Stars
907
Forks
135
Avg merge
11h 28m
Merged PRs (30d)
7

Description

This might sound absurd but I want to have the benefit of mix two tracking modules that can track certain things better. And vice versa:

Quest Pro's lower face tracking can track Nose Sneer, but cannot track tongue directions or individual cheek inflating or suck.
Vive Facial Tracker can track tongue directions and has overall better lower face tracking, but cannot track certain extra blendshapes like nose sneer which Quest Pro tracks.

So how about let us customize what tracked shapes the modules will send over OSC to vrchat? Currently mixing modules might cause interference if two lower face modules are sending data at the same time. So to avoid this kind of issue, add options to customize what blendshapes will the active modules track, so we can combine two or more.
This is because I want to experiment in the future to use my vive facial tracker on my quest pro, but I don't want to lose the nose sneer tracking which is part of the lower face tracking camera of the quest pro

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing how the Quest Pro and Vive Facial Tracker modules produce tracked blendshapes and how active modules send them over OSC to VRChat. Define the customization scope and verify that combining modules avoids interference while preserving selected shapes such as nose sneer, tongue directions, cheek inflating, and cheek suck.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
computer-vision, desktop
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.