huggingface / huggingface/diffusers

EDGE: Editable Dance Generation From Music

Open
#3,817 0 comments 0 reactions 0 assignees View on GitHub
New pipeline/model
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

### Model/Pipeline/Scheduler description

The authors of this paper propose a diffusion-based method for generating human dance sequences (represented by joint angle trajectories) conditioned on music audio embeddings. The weights are stored in Google Docs and are open to the public, but translating the weights into a Diffusers-compatible format will require some surgery. Incorporating this model may also require implementing FiLM timestep conditioning and other possibly currently unavailable NN modules in Diffusers.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available (Only relevant if addition is not a scheduler).

### Provide useful links for the implementation

Paper: https://arxiv.org/pdf/2211.10658.pdf
Website: https://edge-dance.github.io/
Github: https://github.com/Stanford-TML/EDGE

@jtseng20 @Stanford-TML

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.