microsoft / microsoft/winappCli

ui record: element-scoped recording of windowed popups (XAML flyouts/teaching tips) captures the underlying window

Open
#646 0 comments 0 reactions 0 assignees View on GitHub
bug ui-automation
Dominant language
C#
Stars
1.3k
Forks
80
Avg merge
3d 6h
Merged PRs (30d)
51

Description

## Summary

When `winapp ui record ` targets an **element that lives inside a popup which renders in its own top-level window** — e.g. a WinUI/XAML flyout, teaching tip, tooltip, or menu backed by `Xaml_WindowedPopupClass` — the recorder can capture the **underlying main window** instead of the popup, producing blank or stale frames (exit code 0, valid MP4, wrong pixels).

This is a **known limitation** documented in `docs/ui-automation.md` and `docs/usage.md`. Whole-window recording (the primary use case) is unaffected.

## Repro

1. Launch an app that shows a XAML windowed popup (e.g. mspaint's "Got it" teaching-tip, or any WinUI flyout).
2. `winapp ui record -a --duration-sec 3 -o out.mp4`
3. `out.mp4` shows the main window content behind the popup, not the popup itself.

## Root cause

The element-capture retarget logic (`ResolveElementTopLevelHwnd` in `UiAutomationService.Record.cs`) walks the element's **UIA control-view ancestor chain** to find the top-level capture HWND. For a windowed popup, the element renders in a separate HWND but its UIA parent chain leads back to the **main window's** root, so the walk returns the session HWND and no retarget happens — the recorder captures the main window's pixels.

An earlier attempt used `WindowFromPoint` z-order hit-testing to find the popup HWND, but that false-positived on unrelated overlapping windows and was reverted. The robust fix needs to match the element's UIA **RuntimeId** against the app's **owned** popup windows (same-process + owned-by-session) rather than geometry hit-testing.

## Workarounds

- **Record the whole window** (omit the selector): `winapp ui record -a -o out.mp4` — captures everything including the popup.
- **For popup stills**, `winapp ui screenshot --capture-screen` correctly captures windowed popups/overlays.

## Proposed fix (deferred)

In `ResolveElementTopLevelHwnd`, when the UIA ancestor walk resolves to the session HWND, add a constrained fallback: enumerate the target app's **owned** top-level windows (visible, `GW_OWNER` chains to a session window, same PID), and retarget to the one whose UIA subtree contains the element by **RuntimeId** match. Reject foreign-process HWNDs (avoids a capture-redirect vector) and require a unique match (avoids the earlier hit-test false-positive regression).

---

*Filed from an AI-assisted `pr-review` pass on #599. Element-capture geometry has been the recurring edge-case surface in this PR; whole-window recording is solid.*

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in UiAutomationService.Record.cs at ResolveElementTopLevelHwnd and trace the existing UIA ancestor resolution and popup-window handling. Implement the constrained owned-window RuntimeId fallback described in the issue, then verify that element-scoped recording targets the popup while foreign or ambiguous windows are rejected.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
cli, desktop
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.