huggingface / huggingface/diffusers
Add support for JoyAI-Video-Edit
- Lingua principale
- Python
- Stelle
- 34.5k
- Fork
- 7.3k
- Merge medio
- 3g 3h
- PR unite (30g)
- 91
Descrizione
### Model/Pipeline/Scheduler description
[JoyAI-Video-Edit](https://github.com/jd-opensource/JoyAI-Video-Edit) is an instruction-guided video editing model based on the JoyAI streaming architecture.
It performs causal, chunk-wise video denoising with a per-layer KV cache. Each chunk attends to previously denoised chunks and may additionally attend to an optional reference image.
Prompt embeddings are produced by the external [MiMo-VL](https://huggingface.co/XiaomiMiMo/MiMo-VL-7B-RL-2508) model.
### Open source status
- [x] The model implementation is available.
- [x] The model weights are available (Only relevant if addition is not a scheduler).
### Provide useful links for the implementation
- Reference implementation: https://github.com/jd-opensource/JoyAI-Video-Edit
- Original checkpoint: https://huggingface.co/jdopensource/JoyAI-Video-Edit
- MiMo-VL: https://huggingface.co/XiaomiMiMo/MiMo-VL-7B-RL-2508
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
No repository files, tests, or entry points are named. Start by comparing the linked reference implementation and checkpoint with existing video model integrations in diffusers, then determine how MiMo-VL prompt embeddings, causal KV caching, chunk-wise denoising, and optional reference images fit the project. Done means JoyAI-Video-Edit is supported using the linked model resources.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- python, pytorch
- Ambito
- computer-vision, machine-learning
- Tipo di issue
- Funzionalità
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Stato di attività
- Tranquilla
- Chiarezza
- Da chiarire
- Idoneità per principianti
- 35/100