OpenImagingLab / OpenImagingLab/FlashVSR
Question about evaluation details
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.9k
- Forks
- 152
- PR merge metrics
- No merged PRs in 30d
Description
Hi, thanks for your great work!
I have a question regarding evaluation on datasets like SPMCS when using the provided inference code.
When running the provided infer script, the input is forced to 128-multiple resolution and 8n+1 frames. For example:
- 960×540 → 896×512 (center crop)
- Frame count is also reduced after alignment
This leads to mismatched spatial size and frame count compared to the ground truth.
Could you clarify:
- Is this expected behavior for evaluation?
- How do you compute metrics under this setting (e.g., crop GT / pad / other)?
- Would it be possible to release the official evaluation code?
Thanks a lot!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the provided infer script and the SPMCS evaluation setup described in the issue. Determine how the forced spatial and temporal alignment affects metric computation, then document whether the ground truth is cropped, padded, or otherwise adjusted; done means answering the three evaluation questions and clarifying whether official evaluation code can be released.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, testing
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100