Resizing cropped images to 64x64 before processing them with the captioning model results in lower accuracy captions

https://github.com/microsoft/OmniParser/blob/5171b092483ab3e74ca50b9357e225f9f3571f18/util/utils.py#L92

Why exactly is this resize required? Was the model fine-tuned on images that were all scaled to 64x64 in the dataset?

The reason this is causing inaccuracies is that the icon detection model will accurately detect things like text input fields (that are longer in width) but if you scale that text input field to 64x64 (a 1:1 ratio), it just looks like a checkbox and that is exactly what the captioning model detects it as, instead of captioning it as an input or text field it thinks it's a checkbox because it got scaled down to look like a square box.

Does the original base model require that specific size of icons? Can we fine-tune on any size images?

Start at util/utils.py#L92 and trace why cropped images are resized to 64x64 before captioning. Check the model and dataset assumptions about input size and aspect ratio, then compare captions for original and resized crops. Done means the resize requirement and its effect on elongated controls are established, with the issue resolved or the rationale documented.