UITARS-1.5B-7B Grounding error and Screenspot pro score reproduction
- Dominant language
- Python
- Stars
- 11.5k
- Forks
- 877
- PR merge metrics
- No merged PRs in 30d
Description
Hi team, we tried to reproduce the reported ScreenSpot Pro score (48) without success, we used the suggested prompt template and postprocessing, got only around 40.
Also we saw some grounding failure on some simple examples, mostly on predicted X value. This are some examples
1. Click the Open button
2. Click the "Absence" app
Could you please test on your side as well? Thank you.
Contributor guide
No contributing guide indexed for this repository
Research direction
No file, test, or entry point is named. Start by running the suggested prompt template and postprocessing on the two reported examples and the ScreenSpot Pro evaluation; done means documenting whether the score of 48 and the X-coordinate grounding failures can be reproduced.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, machine-learning
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100