OpenImagingLab / OpenImagingLab/FlashVSR
How do you align output and GT for reference metrics when sizes differ?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.9k
- Forks
- 152
- PR merge metrics
- No merged PRs in 30d
Description
Thanks for the great work and for releasing the code.
I have a question about the evaluation protocol for the full-reference metrics, specifically how output/GT resolution mismatch is handled.
The inference pipeline requires the network input to be a multiple of 128 in each spatial dimension. As far as I can tell,
examples/WanVSR/infer_flashvsr_v1.1_tiny.py
tW = (sW // multiple) * multiple # multiple = 128
tH = (sH // multiple) * multiple
upscale_then_center_crop(...)
Because of this floor-crop, the SR output is not the same size as the original ground truth. For example, on REDS (GT 1280×720):
LR input: 320×180
×4 upscale: 1280×720
floor to multiple of 128: 1280×640 (width 1280 is already 10×128, but height 720 → 640)
So the output loses 80 px in height relative to the GT.
Question
The paper reports PSNR/SSIM/LPIPS on YouHQ40, REDS, and SPMCS, but I couldn't find how the output and GT are spatially aligned before computing these pixel-aligned metrics. When the output size (e.g. 1280×640) differs from the GT size (e.g. 1280×720):
How do you align them for the reference metrics — do you center-crop the GT to the output size, pad the output back, or something else?
Are the reported PSNR/SSIM/LPIPS computed on the cropped region only, or on the full GT frame?
If baselines produce full-resolution outputs while FlashVSR is evaluated on a cropped region, how is a fair comparison ensured?
Could you share the exact evaluation alignment step (ideally the script) so the numbers can be reproduced and compared fairly? Thanks!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with examples/WanVSR/infer_flashvsr_v1.1_tiny.py and trace how the output dimensions are produced after the multiple-of-128 crop. Search the repository for the PSNR, SSIM, and LPIPS evaluation entry points; done means the spatial alignment step and evaluation region are documented well enough to reproduce and compare the reported metrics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, testing
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100