airo-ugent / airo-ugent/airo-mono

Evaluate how images are resized in the repo

Aberta
#89 3 comentários 0 reações 1 responsável Reivindicada por @Victorlouisdg Ver no GitHub
camera-toolkit enhancement
Linguagem predominante
Python
Estrelas
32
Forks
8
Merge médio
8min
PRs com merge (30d)
4

Descrição

Recently I've discovered that `cv2.resize` and `PIL.Image.resize` have quite different implementations.

In particular, opencv does not scale the size of the filter used in downscaling (i.e. even if you downscale 100 times, only the neighbouring pixels are used to calculate the interpolation), which results in aliasing artifacts. Most notably, sharp edges will have 'staircase' artifacts, which increase with the size of the downscaling. (funny sidenote downscaling twice 2x will give better results than downscaling once 4x). Pillow uses adaptive filter size, which avoids aliasing by 'smoothing' the image. The downscaled image will contain no artifacts but will be a bit 'blurred'.

More information: https://arxiv.org/pdf/2104.11222.pdf, https://zuru.tech/blog/the-dangers-behind-image-resizing
illustration:
![image](https://github.com/airo-ugent/airo-mono/assets/37955681/e153c633-72f7-420b-beff-5dc47508425b)

This also impacts Deep learning training. I've found that mAP can _differ up to 5%_ when resizing with PIL on the train set and with cv2 on the test set versus using the same resize method for both (downscaling with factor 4).

Long story short, we should probably make sure that we consistently use the same downsampling method as much as possible in the repo to make sure that the artifacts do not influence inference/test performance.

I've now used **PILLOW's bicubic interpolation** in the coco tools and suggest to set that as default throughout the repo.

@Victorlouisdg, the image_transforms in the camera toolkit is the first place in the repo that comes to mind but there might be other places where we downscale images.

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.