feat(linux): add Linux computer-use backend (AT-SPI/X11 accessibility)
- Dominant language
- TypeScript
- Stars
- 5.4k
- Forks
- 502
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 715
Description
## Summary
The Computer Use subsystem (packages/computer-use) is entirely macOS-native. selectComputerUseBackend() returns NONE for any non-Darwin platform, and the maka-cu executor references macOS-specific APIs (AXRaise, AXMenu, ScreenCaptureKit, macOS seatbelt).
## Current State
- **packages/computer-use/src/select-backend.ts:106**: Hardcoded if (process.platform !== 'darwin') return NONE
- **packages/computer-use/src/maka-cu-backend.ts**: 1300+ lines of macOS-specific code
- **pps/desktop/src/main/computer-use/**: 7 files with macOS-only Electron desktop features (PIP windows, cursor overlay, screen lock)
## Proposed Solution
1. Create a platform abstraction layer in packages/computer-use to decouple macOS-specific logic
2. Implement a Linux backend using AT-SPI (Assistive Technology Service Provider Interface) for accessibility
3. Add X11/Wayland screen capture support
4. Add desktop fallback implementations for Windows/Linux (no-op or alternative)
## Acceptance Criteria
- [ ] Platform abstraction layer allows pluggable backends
- [ ] Linux backend can perform basic screen capture and element inspection
- [ ] Desktop computer-use components have platform guards with graceful fallbacks
- [ ] Tests run on Linux without skipping computer-use tests
## Labels
enhancement, help wanted
Contributor guide
Research direction
Start by reading packages/computer-use/src/select-backend.ts and packages/computer-use/src/maka-cu-backend.ts, then inspect the seven files under apps/desktop/src/main/computer-use/. Compare the current macOS-only paths with the acceptance criteria; done means a pluggable Linux backend, basic screen capture and element inspection, guarded desktop fallbacks, and Linux tests that no longer skip computer-use coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- electron, linux, typescript
- Domain
- accessibility, desktop, operating-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100