Lightning-AI / Lightning-AI/torchmetrics
Upper Face Dynamic Deviation (FDD) Metrics for 3D Talking Heads Evaluation
@rittik9 is already working on this.
Since Jul 11, 2025.
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 526
- Avg merge
- 6d 11h
- Merged PRs (30d)
- 5
Description
## 🚀 Feature
I would like to propose incorporating an essential evaluation metric for 3D talking heads into the TorchMetrics library: Upper Face Dynamic Deviation (FDD).
### Motivation
Current TorchMetrics offerings lack dedicated metrics for evaluating 3D talking heads, except for [LVE](https://github.com/Lightning-AI/torchmetrics/issues/3003). I think this metric also fits in multimodal folder of this library.
### Pitch
This metric is widely used in speech-driven facial animation research, it measures the variation of facial dynamics for motion sequences in comparison with ground truth. It gives an indication of how close the standard deviation (or upper face motion variation) of generated sequences (of test-set audios) is compared to the variation observed in ground truth.
### Reference
- Paper : [Codetalker](https://openaccess.thecvf.com/content/CVPR2023/papers/Xing_CodeTalker_Speech-Driven_3D_Facial_Animation_With_Discrete_Motion_Prior_CVPR_2023_paper.pdf)
### Additional context
If agreed, I would like to open a PR for the same.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.