huggingface / huggingface/diffusers
Add support for PPVCtrl
- Dominant language
- Python
- Stars
- 34.5k
- Forks
- 7.3k
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 91
Description
### Model/Pipeline/Scheduler description
I recently came across an impressive work called PPVCtrl, a controllable video generation model. It leverages an auxiliary condition encoder to transform a text-to-video generation model into a customizable video generator, all without retraining the original generator. It's akin to ControlNet, but for video generation.
Prompt
Reference Image
Control Videos
PP-VCtrl-5B-T2V
PP-VCtrl-5B-I2V
Group of fishes swimming in aquarium.
A boat with a flag on it is sailing on the sea.
Prompt
Reference Image
Control Videos
PP-VCtrl-5B-T2V
PP-VCtrl-5B-I2V
A rider in a dark helmet and white breeches is atop a chestnut horse...
A dark gray Mini Cooper is parked on a city street...
Prompt
Reference Image
Pose Videos
PP-VCtrl-5B-I2V
A young man with curly hair and a red t-shirt featuring a white logo is seen in various states of motion...
A woman models an Adrianna Papell women's gown, featuring a sleeveless...
### Open source status
- [x] The model implementation is available.
- [x] The model weights are available (Only relevant if addition is not a scheduler).
### Provide useful links for the implementation
https://github.com/PaddlePaddle/PaddleMIX/tree/develop/ppdiffusers/examples/ppvctrl
Contributor guide
Assessment
This issue has not been assessed yet.