bytedance / bytedance/Q-Insight

RALI is not robust against noise from Nano Banana Pro

Open
#16 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
322
Forks
13
PR merge metrics
No merged PRs in 30d

Description

Thanks for open-sourcing this great work! I would like to bring this edge case to your attention:

RALI gives a higher score for a noisy image than for a clean image, when the noise comes from an image editing model.

For example, this clean image receive a score of **3.89**.

Image

However, this noisy image after 5 steps by Nano Banana Pro receives a higher RALI score of **3.93**, despite obvious visual artifacts.

Image

Traditional NR-IQA metrics also fail in this case. Hope to see future improvements of the Q-Insight model family! You can find a dataset of images corrupted by Nano Banana Pro [here](https://huggingface.co/datasets/kenantang/Banana100).

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.