bytedance / bytedance/Video-As-Prompt

The generated result remains unchanged when modifying the reference video

Open
#13 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
454
Forks
32
PR merge metrics
No merged PRs in 30d

Description

Hi,
Thank you for sharing this excellent work!

I have been testing the inference code of wan_vap. When the reference video and the target prompt are consistent, the generated results are fantastic, just as shown in your demo.

However, I noticed that the generated output doesn't seem to change at all when I modify the reference video. Even when I use a completely different reference video, the generated result remains exactly the same. It feels as though the model is ignoring the reference video condition during inference.

Is there a specific configuration, weight, or parameter I need to adjust to enforce the video conditioning?

Thanks in advance for your help!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the wan_vap inference code mentioned in the report and inspect how the reference video is passed into conditioning. Compare runs using different reference videos and configuration, with the goal of confirming that the generated output changes when the reference video changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.