Jordan-Hall / Jordan-Hall/browser
[P0][PC-02] Windows native-control adapter
- Dominant language
- No language data
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Programme: #1
Epic: #23
## Objective
Implement the first production desktop-control backend on Windows using UI Automation and platform APIs, with bounded visual fallback only where necessary.
## Scope
- Window/process enumeration and identity mapping.
- UI Automation tree extraction and stable element references where possible.
- Invoke/select/set-value/text/focus/scroll patterns.
- Application/file launch and handoff through scoped runtime capabilities.
- Bounded screenshot/visual grounding fallback for unsupported controls.
- User takeover detection and physical-input lease integration.
- Handling elevation boundaries, UAC/secure desktop, permission prompts and inaccessible controls.
- Reference workflow corpus across common native/Win32/UWP/Chromium/Electron apps.
## Safety rules
- Never automate secure-desktop approval, UAC elevation, login protection, MFA or CAPTCHA bypass.
- Re-observe state after focus/window changes.
- Visual fallback does not inherit semantic confidence.
## Acceptance criteria
- [ ] Reference Windows workflows complete using semantic UIA where supported and verify postconditions.
- [ ] UAC/secure-desktop and protected auth flows stop for user handoff.
- [ ] Changed/closed/replaced windows invalidate stale targets.
- [ ] Visual fallback is bounded and separately logged/evaluated.
- [ ] Manual user input relinquishes agent control.
- [ ] Accessibility/control failures return structured diagnostics rather than blind retries.
## Dependencies
- PC-01
**First phase:** P0
**Maturity target:** P3
**Owner:** harness-pc-providers
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing dependency PC-01 and the objective, scope, and safety rules for the Windows native-control backend. Use the acceptance criteria as the completion checklist: semantic UI Automation workflows with postcondition checks, safe user handoff for protected flows, stale-target invalidation, bounded visual fallback, input relinquishment, and structured diagnostics.
Written by the indexing model from the issue text.
Assessment
- Domain
- desktop, operating-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100