OpenImagingLab / OpenImagingLab/FlashVSR
missalignment problem
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.9k
- Forks
- 152
- PR merge metrics
- No merged PRs in 30d
Description
Thanks for opening source such great work! I have a problem about the missalignment between the generated video and the input low quality video. According to the largest_8n1_leq function the length of the low quallity video is 8n+1, while the length of the noise input to the diffusion model is 2n, corresponding to 8n-3 frames of the generated video. Moreover, the first 6 noise takes first 25 frames of low quality video as condition and each subsequent noise corresponds to 4 frames of low-resolution video. 8n+1 frames of low quality video are needed as input to the diffusion model. While when the latents are converted into video by the TCDecoder, only 8n-3 frames of low quality video are needed. Why is there a missalignment between the input video and the generated video?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Trace the frame indexing from largest_8n1_leq in examples/WanVSR/infer_flashvsr_v1.1_tiny.py, then compare it with the TCDecoder call in diffsynth/pipelines/flashvsr_tiny.py. Reproduce the stated 8n+1 versus 8n-3 inputs and determine whether the difference is intentional or an alignment bug. Done means the frame mapping is explained and any required correction is covered by a reproducible check.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100