anthropics / anthropics/claude-quickstarts

Feature Request: Add partial/window screenshot support to cut latency & token usage

Open
#283 0 comments 0 reactions 0 assignees View on GitHub
computer use enhancement
Dominant language
TypeScript
Stars
17.6k
Forks
3k
Avg merge
5h 13m
Merged PRs (30d)
3

Description

First of all, huge thanks for releasing the computer-use-demo. We genuinely enjoy using it and are excited to roll it out more intensively in our
internal testing environment.

**The Problem:**
Inside a Ubuntu 22.04 virtual machine (but the same happens when the demo is run via Docker Compose), I’m using the computer-use-demo agent to analyse a single app window.
The built-in screenshot() helper always captures the entire desktop, even when the target window covers only ~25 % of the screen. Converting that full-screen PNG to base-64 and streaming it to Claude:
• adds ~3–4 s to every analysis cycle
• inflates prompt size by 3 × – 4 × (unnecessary token cost)

**Desired behaviour**
• Partial (region) capture – supply x, y, width, height or a bounding-box tuple.
• Active-window capture – supply the window name.
Either mode would trim the image to the relevant context, speeding up cycles and shrinking the base-64 payload dramatically.

**Alternative Solution**
If adding this directly isn’t on the roadmap, could you please:
• Document the recommended extension points for the computer tool, especially around customizing the screenshot() helper, or
• Expose a simple plug-in hook (e.g., ComputerMixin or env-var override) so we can swap in a custom screenshot command without patching the library.

Thanks again for the awesome tool!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.