microsoft / microsoft/OmniParser
How to collect bounding boxes of interactable regions of the webpage from the DOM tree ?
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 25.4k
- Forks
- 2.2k
- PR merge metrics
- No merged PRs in 30d
Description
Thank you for the excellent work ！！！ In the paper, I find some describtion " Specifically, we curate a dataset of interactable icon detection dataset, containing 67k unique screenshot images, each labeled with bounding boxes of interactable icons derived from DOM tree. We first took a 100k uniform sample of popular publicly availabe urls on the web [OXL+22], and collect bounding boxes of interactable regions of the webpage from the DOM tree of each urls. Some examples of the webpage and the interactable regions are shown in 2. " 。
I want to know how to obtain relevant location information for bbox with clickable elements in Dom Tree ?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue discusses the paper’s DOM-tree-derived bounding boxes for clickable webpage elements but names no repository file, test, or entry point. Start by locating any dataset-generation or DOM-collection material in the repository; done would be a documented explanation of how the clickable elements’ locations are obtained.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- html
- Domain
- computer-vision, web-dev
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100