Shared-machine coexistence: run all day alongside an active human
- Dominant language
- Rust
- Stars
- 41k
- Forks
- 3.6k
- Avg merge
- 13h 59m
- Merged PRs (30d)
- 299
Description
Operating Computer Use on the same Mac a human is actively using is disruptive today (observed during the 0.8.0 muse acceptance pass, Hmbown/Codewhale#6300): agent windows appear on the human's screen, apps launch visibly, every window-record key takes a front-process lease that can swallow or misdirect concurrent human keystrokes, and one background re-bind observed the scratch app frontmost mid-session. Observation (AX queries, screenshots, menus) is invisible; input leases and visible artifacts are the cost.
Gaps, grouped:
1. Hide the work: launch hidden, keep agent windows off the human's Space / ordered back; clean up everything created. Ceiling note: macOS has one front process per user session, so a second display does not isolate input. Sustained autonomous work belongs on a separate computer (VM / cloud / second Mac via the existing registry); same-screen is for brief idle-gated assists.
2. Yield to the human: idle-gating (lease only when idle N seconds), yield-on-input (abort lease on HID activity), dispatch-time frontmost revalidation, lease-free routes (AX actions) always preferred over window-record.
3. Honest interference accounting: receipts report lease duration plus whether user input occurred during the lease; per-day audit log of what the agent touched, per app.
4. Human controls: global working indicator with one-click pause, quiet hours, per-app hands-off list. Make "do not work in apps the user is actively editing" system-enforced (frontmost / recent-input signals with refuse-or-escalate), not model advice.
5. Contention correctness: agent transfers via a private pasteboard with human-clipboard restore + verify; no stale-input replay after pause; safe no-op when the human switches windows between observe and act.
6. Scheduling and manners: batch disruptive actions; reads anytime, writes when idle; rate-limit leases.
Acceptance sketch: a human types continuously in their own apps while the agent runs an hour-long task in its own apps — zero lost or misdirected keystrokes, zero focus thefts, zero agent windows on the human's Space; the audit log shows every lease with duration and zero user-input-during-lease flags. Pause and the hands-off list are honored in flight (<1s).
Related: Hmbown/Codewhale#6300 (acceptance pass where this was observed), Hmbown/Codewhale#5927 (wrong-target input incident class), Hmbown/Codewhale#5856 (Engine acceptance incl. no stale replay).
Contributor guide
Research direction
No files, tests, or entry points are named. Start by mapping the existing Computer Use window, input-lease, pasteboard, pause, and audit-log paths, then review the related issues for constraints and prior incidents. Done means the stated acceptance sketch passes: no lost input or focus theft, no windows on the human’s Space, accurate lease receipts, and pause and hands-off controls honored in flight.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- macos, rust
- Domain
- desktop, operating-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100