facebookresearch / facebookresearch/av_flow

No download link

Open
#1 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
25
Forks
2
PR merge metrics
No merged PRs in 30d

Description

Hello, this is a very good job, but I have some questions. Firstly, there is no download link for the image here.:

Download the participant images from:

Move them under: /path/to/dataset/50h_dataset/participant/camera01/images/

To visualize the results, run the following:

python recipes/flow_matching_audiovisual.py -i /path/to/dataset/50h_dataset/ -o /path/to/output/dir/av_flow_50h_dyadic/ -c /path/to/output/dir/av_flow_50h_dyadic/checkpoints/iter-1000000.pth --test
The results (3 generated audio samples and corresponding videos) are saved under /path/to/output/dir/test/segment/viz/.

Secondly, it is mentioned in the paper that Smirk's feature is exp (50)+jaw (3)+pose (3)=56 dimensions, but in the readme it is written as 58 dimensions.

DEFAULT_CONFIG = {
...
"participant_cond": True,
"other_dim": 58 + 29, # 58 SMIRK features and 29 ASR tokens
...
}

I am not sure if it is due to my misunderstanding or if you have written it incorrectly here.
Looking forward to your reply.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.