bytedance / bytedance/UI-TARS

drag 方向问题

Open
#231 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
11.5k
Forks
877
PR merge metrics
No merged PRs in 30d

Description

Action: [{'reflection': None, 'thought': '我看了看手机屏幕,发现刚才的滑动方向不太对。要找到Simple SMS Messenger,我得换个方向,向下滑动屏幕来查看更多应用程序。毕竟
要完成重发消息的任务,首先得找到并打开正确的应用才行。', 'action_type': 'drag', 'action_inputs': {'start_box': ('897', '1208'), 'end_box': ('244', '1208')}, 'text':
"Thought: 我看了看手机屏幕,发现刚才的滑动方向不太对。要找到Simple SMS Messenger,我得换个方向,向下滑动屏幕来查看更多应用程序。毕竟要完成重发消息的任务,首先得找到
并打开正确的应用才行。\nAction: drag(start_box='(897,1208)', end_box='(244,1208)')"}]

在andorid world评估时候发现,model 说的是向下滑动屏幕但是实际上是在水平方向滑动,这个问题如何解决

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the drag action in the AndroidWorld evaluation described in the issue, using the provided start and end coordinates. Trace how the model's “向下滑动” instruction becomes the drag coordinates. Done means a downward instruction produces a vertical drag rather than the reported horizontal swipe.

Written by the indexing model from the issue text.

Assessment

Tech stack
android, python
Domain
ai, mobile-dev
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.