huggingface / huggingface/diffusers

EDGE: Editable Dance Generation From Music

Aperta
#3,817 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
New pipeline/model
Lingua principale
Python
Stelle
34.5k
Fork
7.3k
Merge medio
3g 3h
PR unite (30g)
91

Descrizione

### Model/Pipeline/Scheduler description

The authors of this paper propose a diffusion-based method for generating human dance sequences (represented by joint angle trajectories) conditioned on music audio embeddings. The weights are stored in Google Docs and are open to the public, but translating the weights into a Diffusers-compatible format will require some surgery. Incorporating this model may also require implementing FiLM timestep conditioning and other possibly currently unavailable NN modules in Diffusers.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available (Only relevant if addition is not a scheduler).

### Provide useful links for the implementation

Paper: https://arxiv.org/pdf/2211.10658.pdf
Website: https://edge-dance.github.io/
Github: https://github.com/Stanford-TML/EDGE

@jtseng20 @Stanford-TML

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Start with the linked EDGE GitHub repository and paper to understand the model implementation, then inspect the publicly available weights referenced in the issue. Determine the work needed to translate the weights into a Diffusers-compatible format and whether FiLM timestep conditioning or other unavailable modules are required. Done means the EDGE model is incorporated with its required components and can generate dance sequences conditioned on music.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python, pytorch
Ambito
machine-learning
Tipo di issue
Funzionalità
Difficoltà
5/5
Tempo stimato
Più di una settimana
Stato di attività
Ferma
Chiarezza
Da chiarire
Idoneità per principianti
25/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.