GUI agent capability?
Open
- Dominant language
- No language data
- Stars
- 861
- Forks
- 51
- Avg merge
- 1m
- Merged PRs (30d)
- 1
Description
What about its ability as a GUI agent?
(1) given a GUI task, history actions (including latest history screenshots) and current screenshot, it good at planning next best action to complete the GUI task (e.g., task: booking a flight from Shenzhen to Beijing on August 4 on https://us.trip.com/)?
(2) Can it accurately locate elements (e.g., buttons) on the web UI?
Was related data also included during training?
I think for (1), there must be strong reasoning ability, not sure if model was also trained with this type of data?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.