bytedance / bytedance/UI-TARS

UI-TARS-1.5-7By is easy to enter an endless loop on mobile

Open
#130 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
11.5k
Forks
877
PR merge metrics
No merged PRs in 30d

Description

We made few tests, the agent was easy to get stuck in endless retries, for example:

`Thought: 咦,怎么进到相机界面来了?这可不是我要的。我得赶紧回到抖音主页面继续搜索小冰的视频。看来需要点击一下左上角那个返回按钮,让我们回到正确的轨道上。\nAction: click(start_box='(47,86)')

Thought: 看来我刚才点错了位置,不小心进入了相机界面。不过没关系,我知道该怎么做 - 点击左上角的返回按钮就能回到抖音主页面。这样我们就能继续完成搜索小冰视频的任务了。\nAction: click(start_box='(47,215)')

Thought: 唉,我又不小心点到相机界面了。不过没关系,我知道怎么纠正 - 点击左上角的返回按钮就能回到抖音主页面。这样我就能继续完成搜索小冰视频的任务了。\nAction: click(start_box='(47,215)')

Thought: 我发现自己又误入了相机界面,这可不行。让我想想,要完成搜索小冰视频的任务,我得先回到抖音主页面。看到左上角那个返回按钮了,点击它就能回到正确的页面继续操作。\nAction: click(start_box='(47,215)')

Thought: 我发现自己在相机界面里打转,这可不行。刚才点击返回按钮的尝试都没有成功,看来得换个方式了。既然要回到抖音主页面,我决定直接点击左上角的返回按钮试试看。\nAction: click(start_box='(47,215)')

Thought: 我发现自己在相机界面里打转,这可不行。刚才点击返回按钮的尝试都没有成功,看来得换个方式了。既然要回到抖音主页面,我决定直接点击左上角的返回按钮试试看。\nAction: click(start_box='(47,215)')

`

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the reported mobile scenario and inspect the agent behavior around repeated actions. Use the example trace as the failing case: the agent should not remain in endless retries after repeated taps in the camera interface. The issue does not name a source file or test to run.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, mobile
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.