About Android world's reviews
- Dominant language
- Python
- Stars
- 11.5k
- Forks
- 877
- PR merge metrics
- No merged PRs in 30d
Description
Congratulations, your new model ui-tars 1.5 is a big improvement, and I'd like to reproduce the results of your paper on Android world, and let's ask you how you evaluate it, and what's the prompt like. My rating on the ui-tars-7b-dpo was only 20+, and I didn't improve on the ui-tars 1.5 7b model, wondering if it was a request formatting issue or something. Here's my request format, five pictures in total, four historical records, and all of them. I can give you advice on how you feel. Thank you very much.
"model": self.model,
"temperature": self.temperature,
"top-p": 1,
"top-k": 1,
"messages": [
{
"role": "system",
"content": [{"type": "text","text": "You are a helpful assistant."}]
},
{
"role": "user",
"content": [{"type": "text","text": "prompt + instruction"}]
},
{
"role": "assistant",
"content": "Thought: xxxx。 \n Action: xxxx)"
},
{
"role": "assistant",
"content": "Thought: xxxx。 \n Action: xxxx)"
},
..
..
..
..
{
"role": "user",
"content": [{"type": "image_url","image_url": {"url": "base64"}}]
},
{
"role": "assistant",
"content": "Thought: xxxx。 \n Action: xxxx)"
},
{
"role": "user",
"content": [{"type": "image_url","image_url": {"url": "base64"}}]
}
]
}
]
prompt=“You are a GUI agent. You are given a task and your action history, with screenshots. You need to perform the next action to complete the task.\n\n## Output Format\n\n```text\nThought: ...\nAction: ...\n```\n\n## Action Space\n\n- open_app(app_name='app_name', activity='MainActivity') # open an app with the name stored in the 'app_name' field, and 'activity'. This name should map to an packege name and activity which can be used to open the app using adb.\n\n- click(start_box='<|box_start|>(x1,y1)<|box_end|>') # click at a specific point on the screen using the coordinates '(x1,y1)'.\n\n- long_press(start_box='<|box_start|>(x1,y1)<|box_end|>', time='') # long press at a specific point on the screen using the coordinates '(x1,y1)'.\n\n- type(content='') # types the text in the 'content' field. It does not perform a click or press enter at the end.\n\n- scroll( start_box='<|box_start|>(x1,y1)<|box_end|>', end_box='<|box_start|>(x2,y2)<|box_end|>') # scroll from point '( x1,y1)' to point '(x2,y2)'.\n\n- press_home() # go to the home screen.\n\n- press_back() # go back to the previous screen.\n\n- press_enter() # press `enter` on the keyboard.\n\n- wait(time='') # wait a set number of seconds.\n\n- finished(content='') # Submit the task regardless of whether it succeeds or fails. \n\n## Note\n\n- Use English in `Thought` part.\n\n- Write a small plan and finally summarize your next action (with its target element) in one sentence in `Thought` part.\n\n## Instruction \n”
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no files, tests, or entry points. Begin with the supplied request format and Android World evaluation details; compare the message history, image inputs, sampling parameters, and prompt, then document whether a formatting difference explains the score or provide a reproducible cause.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- android, python
- Domain
- machine-learning, mobile-dev
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100