microsoft / microsoft/OmniParser
Giving more context on image requirements
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 25.4k
- Forks
- 2.2k
- PR merge metrics
- No merged PRs in 30d
Description
Hi everyone,
Having a few questions on the image requirements:
-
What's the minimum device pixel ratio for image that can be passed to get proper results? Currently we send screenshots from retina displays, which are pretty big, and I'm not sure if it has to be that big. Furthermore, maybe it can be even smaller than the 1x device pixel ratio (e.g. 0.5).
-
How significantly does the image dimensions impact the inference time (if anyone did some benchmarking)? E.g. 2x size increases the inference time only by 10% etc.
-
Are there already some optimisations in the OmniParser code that reduce the image dimensions?
Best,
Nikita
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file, test, or entry point is named. Start by locating the image-input and inference code in OmniParser, then benchmark different image dimensions and check for existing resizing optimizations. Done means documenting the minimum device pixel ratio, the effect of dimensions on inference time, and any current dimension-reduction behavior.
Written by the indexing model from the issue text.
Assessment
- Domain
- computer-vision, machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100