microsoft / microsoft/winappCli
ui record: element-scoped recording of windowed popups (XAML flyouts/teaching tips) captures the underlying window
- Dominant language
- C#
- Stars
- 1.3k
- Forks
- 80
- Avg merge
- 3d 6h
- Merged PRs (30d)
- 51
Description
## Summary
When `winapp ui record ` targets an **element that lives inside a popup which renders in its own top-level window** — e.g. a WinUI/XAML flyout, teaching tip, tooltip, or menu backed by `Xaml_WindowedPopupClass` — the recorder can capture the **underlying main window** instead of the popup, producing blank or stale frames (exit code 0, valid MP4, wrong pixels).
This is a **known limitation** documented in `docs/ui-automation.md` and `docs/usage.md`. Whole-window recording (the primary use case) is unaffected.
## Repro
1. Launch an app that shows a XAML windowed popup (e.g. mspaint's "Got it" teaching-tip, or any WinUI flyout).
2. `winapp ui record -a --duration-sec 3 -o out.mp4`
3. `out.mp4` shows the main window content behind the popup, not the popup itself.
## Root cause
The element-capture retarget logic (`ResolveElementTopLevelHwnd` in `UiAutomationService.Record.cs`) walks the element's **UIA control-view ancestor chain** to find the top-level capture HWND. For a windowed popup, the element renders in a separate HWND but its UIA parent chain leads back to the **main window's** root, so the walk returns the session HWND and no retarget happens — the recorder captures the main window's pixels.
An earlier attempt used `WindowFromPoint` z-order hit-testing to find the popup HWND, but that false-positived on unrelated overlapping windows and was reverted. The robust fix needs to match the element's UIA **RuntimeId** against the app's **owned** popup windows (same-process + owned-by-session) rather than geometry hit-testing.
## Workarounds
- **Record the whole window** (omit the selector): `winapp ui record -a -o out.mp4` — captures everything including the popup.
- **For popup stills**, `winapp ui screenshot --capture-screen` correctly captures windowed popups/overlays.
## Proposed fix (deferred)
In `ResolveElementTopLevelHwnd`, when the UIA ancestor walk resolves to the session HWND, add a constrained fallback: enumerate the target app's **owned** top-level windows (visible, `GW_OWNER` chains to a session window, same PID), and retarget to the one whose UIA subtree contains the element by **RuntimeId** match. Reject foreign-process HWNDs (avoids a capture-redirect vector) and require a unique match (avoids the earlier hit-test false-positive regression).
---
*Filed from an AI-assisted `pr-review` pass on #599. Element-capture geometry has been the recurring edge-case surface in this PR; whole-window recording is solid.*
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in UiAutomationService.Record.cs at ResolveElementTopLevelHwnd and trace the existing UIA ancestor resolution and popup-window handling. Implement the constrained owned-window RuntimeId fallback described in the issue, then verify that element-scoped recording targets the popup while foreign or ambiguous windows are rejected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- cli, desktop
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100