bytedance / bytedance/UI-TARS

UI-TARS-1.5-7B Endless Loops on Web Interfaces

Open
#131 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
11.5k
Forks
877
PR merge metrics
No merged PRs in 30d

Description

We've observed that UI-TARS-1.5-7B agent frequently gets stuck in endless retry loops when interacting with web interfaces. The agent repeatedly attempts the same ineffective action, unable to adapt or learn new strategies.

Example of observed behavior:

```
Thought: I see a search button at the top of the page, it's the magnifying glass icon in the upper right corner. To find information about Apple Pencil, I need to click this search button to open the search box.
Action: click(start_box='(1099,64)')
Thought: I see a search button at the top of the page, it's the magnifying glass icon in the upper right corner. To find information about Apple Pencil, I need to click this search button to open the search box.
Action: click(start_box='(1099,64)')
Thought: I see a search button at the top of the page, it's the magnifying glass icon in the upper right corner. To find information about Apple Pencil, I need to click this search button to open the search box.
Action: click(start_box='(1099,64)')

[This exact same Thought and Action repeats 15x times without change]
```

We also need deployment guidelines for web operations, including:
1. Recommended screen resolution settings
2. Input prompt formatting and concatenation methods for web tasks
3. Specific configuration steps for web deployment

Please provide relevant documentation or consider creating web-specific deployment guidelines.

Contributor guide

No contributing guide indexed for this repository

Research direction

No source files, tests, or entry points are named. Start by reviewing the web deployment and prompt-handling entry points used by UI-TARS-1.5-7B, then reproduce the repeated-action behavior. Done means the loop behavior is addressed or scoped, with web deployment guidance covering resolution, prompt formatting, concatenation, and configuration.

Written by the indexing model from the issue text.

Assessment

Domain
ai, web-dev
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.