KhronosGroup / KhronosGroup/UnityGLTF

Facial blendshapes not driven after applying Meta face tracking scripts to GLB imported via UnityGLTF

Open
#852 1 comment 0 reactions 1 assignee Claimed by @robertdorn83 View on GitHub
Dominant language
C#
Stars
2.2k
Forks
536
PR merge metrics
No merged PRs in 30d

Description

### Describe the bug 💬

I'm encountering an issue when instantiating a .glb file using the UnityGLTF loader from KhronosGroup. After loading the model, I apply Meta's face tracking setup script using:

`AddComponentsHelper.SetUpCharacterForA2EARKitFace(gameObject, false, true);`

The script components are correctly added to the avatar, but the face tracking does not animate or affect the blendshapes of the model. The blendshapes remain static, even though the tracking pipeline appears to be active.

I tested the exact same process using GLTFast instead of UnityGLTF, and in that case, everything worked as expected — the facial expressions were correctly applied through the blendshapes. However, due to some package constraints, I prefer using UnityGLTF.

### Steps to reproduce 🔢

Steps to Reproduce:

1. Instance a .glb avatar using UnityGLTF.
2. Apply SetUpCharacterForA2EARKitFace(...) from Meta's Movement SDK.
3. Run the scene with active face tracking (using a OVRCameraRig, from the All-In-One-SDK of Meta, with "Record Audio for audio based Face Tracking"). Or Just a script that after instancing the model modifies its blendshapes
4. Build and run the app on a Quest 1, 2 or 3 device

![Image](https://github.com/user-attachments/assets/c505b816-77a1-44a5-93bc-6c6b10b8865d)

Observe that no blendshape animation occurs.

### Files to reproduce the issue ♻

[ModelForTesting.zip](https://github.com/user-attachments/files/20383593/ModelForTesting.zip)

### Editor Version 🎲

6000.0

### Render Pipeline and version

URP 17.0.4

### UnityGLTF Version

2.14.1

### Operating System 👩‍💻

Windows

### When does this problem happen?

- [ ] Editor Import
- [x] Runtime Import
- [ ] Editor Export
- [ ] Runtime Export

### Additional Info 📜

_No response_

### Validations 🩹

- [x] I have [searched existing issues](https://github.com/KhronosGroup/UnityGLTF/issues): no issue already exist that reports the same problem.
- [x] I follow the [Code of Conduct](https://www.khronos.org/about/code-of-conduct)
- [x] I provided a [minimal reproducible example](https://stackoverflow.com/help/minimal-reproducible-example), including files when necessary.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.