benaclejames / benaclejames/VRCFaceTracking

LipTrackingActive and EyeTrackingActive parameters are not reliable at all

Open
#277 1 comment 0 reactions 0 assignees View on GitHub
bug enhancement
Dominant language
C#
Stars
907
Forks
135
Avg merge
11h 28m
Merged PRs (30d)
7

Description

this is a copy paste from a thread in the discord, to keep this in a more visible place https://discord.com/channels/849300336128032789/1369298092184047616

according to the docs, "These parameters are set by VRCFaceTracking to indicate the status of the relevant tracking module. They are only sent upon load or when the status of the module changes." - https://docs.vrcft.io/docs/tutorial-avatars/tutorial-avatars-extras/parameters#tracking-active-parameters.

however, there seems to be multiple issues with them:

when you first load into VRChat, even if both eye and lip tracking modules are loaded, sometimes one or both parameters fail to get set.
if the parameters are marked are not marked as Saved, they will not be set properly when the user changes avatar or switches worlds, and face tracking won't work. if they are marked as Saved, and the user closes VRChat, the next time they launch the game, the parameters can be True even if VRCFT isn't running. this means the value of the parameters can't be trusted to enable/disable functionality.
the docs say to not use them in the expression menu, however, it seems like Jerry's templates do use them for enabling or disabling eye/lip tracking. this complicates things a bit, and means that changing the behavior of the parameters may have consequences.

the reason I bring this up is because I'm working on a better viseme replacement system that works in conjunction with face tracking to improve the responsiveness of speech, with additional logic to avoid over-driving shapes. if the user currently isn't using lip tracking, I need to know that so I can make the visemes have full influence, but I can't reliably do that with the current behavior of the parameters. if the parameters worked as stated in the docs, this wouldn't be a problem.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the linked documentation and Discord thread, then compare the reported load, avatar/world switch, and saved-parameter behavior with the documented tracking-status lifecycle. Done should mean the parameters reliably represent whether eye and lip tracking modules are active, while accounting for the stated effects on expression-menu templates and viseme systems.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
desktop
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.