bytedance / bytedance/UI-TARS

UITARS-1.5B-7B Grounding error and Screenspot pro score reproduction

Open
#215 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
11.5k
Forks
877
PR merge metrics
No merged PRs in 30d

Description

Hi team, we tried to reproduce the reported ScreenSpot Pro score (48) without success, we used the suggested prompt template and postprocessing, got only around 40.

Also we saw some grounding failure on some simple examples, mostly on predicted X value. This are some examples

1. Click the Open button

Image

2. Click the "Absence" app
Image

Could you please test on your side as well? Thank you.

Contributor guide

No contributing guide indexed for this repository

Research direction

No file, test, or entry point is named. Start by running the suggested prompt template and postprocessing on the two reported examples and the ScreenSpot Pro evaluation; done means documenting whether the score of 48 and the X-coordinate grounding failures can be reproduced.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.