[maui-labs docs] Document ScreenshotResultAsync API and improved macOS screenshot reliability (PR #349)
- Dominant language
- No language data
- Stars
- 282
- Forks
- 265
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 19
Description
## Source PR
**PR**: https://github.com/dotnet/maui-labs/pull/349
**Title**: Fix non-retryable macOS screenshot failure when window not frontmost
**Author**: Jonathan Dick (`@jondick`)
**Merged**: 2026-07-08
---
## Summary of Changes
This PR introduces two user-facing improvements to the DevFlow screenshot feature:
1. **New `AgentClient.ScreenshotResultAsync()` public API** in `Microsoft.Maui.DevFlow.Driver` — a richer alternative to the existing `ScreenshotAsync()` that returns a structured `ScreenshotResult` object instead of a nullable `byte[]`. The result carries success/failure status, the PNG bytes on success, and on failure: a human-readable error, a machine-readable reason code (e.g. `window-not-frontmost`), a `Retryable` flag, and optional actionable suggestions.
2. **macOS screenshot reliability improvement** — Screenshots now work even when the MAUI app is not the frontmost/active application. Previously this always returned null (silent failure). The new behavior tries multiple capture strategies (CGWindowListCreateImage → CacheDisplay → DataWithPdfInsideRect). If all fail, a structured error is returned with `Retryable: true` and suggestions (e.g. "Bring the app to the foreground").
3. **Improved `maui_screenshot` MCP tool error messages** — The tool now surfaces actionable error text and suggestions to AI agents when a screenshot fails, rather than a generic exception.
---
## Documentation Pages Affected
- **DevFlow Driver / AgentClient API reference**: `docs/developer-tools/devflow/` — needs to document the new `ScreenshotResultAsync` method and the `ScreenshotResult` type.
- **MCP Tools reference** (if a `maui_screenshot` tool page or table exists): update to note improved error messaging and macOS reliability.
- **Known issues / troubleshooting** (if any): the macOS "window not frontmost" scenario may be worth calling out explicitly.
---
## Suggested Changes
### 1. AgentClient API reference — add `ScreenshotResultAsync`
Wherever `ScreenshotAsync` is documented (e.g. a method table or API reference page for `Microsoft.Maui.DevFlow.Driver.AgentClient`), add a new row/section:
````markdown
### `ScreenshotResultAsync`
```csharp
Task(ScreenshotResult) ScreenshotResultAsync(
int? window = null,
string? elementId = null,
string? selector = null,
int? maxWidth = null,
string? scale = null)
```
Captures a screenshot and returns a `ScreenshotResult` with structured success or failure
information. Prefer this over `ScreenshotAsync` when you need actionable error details.
| Property | Type | Description |
|----------|------|-------------|
| `Success` | `bool` | `true` if the capture succeeded. |
| `Data` | `byte[]?` | PNG bytes when `Success` is `true`. |
| `Error` | `string?` | Human-readable error message when `Success` is `false`. |
| `Reason` | `string?` | Machine-readable cause code (e.g. `window-not-frontmost`). |
| `Retryable` | `bool` | `true` if retrying after correcting the condition (e.g. foregrounding the app) may succeed. |
| `Suggestions` | `IReadOnlyList(string)?` | Actionable suggestions from the agent. |
**Example**
```csharp
var result = await agentClient.ScreenshotResultAsync();
if (result.Success)
{
File.WriteAllBytes("screenshot.png", result.Data!);
}
else
{
Console.WriteLine(result.Error);
if (result.Retryable)
Console.WriteLine("Tip: " + string.Join(", ", result.Suggestions ?? []));
}
```
````
### 2. MCP `maui_screenshot` tool reference
If a table or page describes `maui_screenshot`, add a note:
````markdown
> **Note**: When a screenshot cannot be captured (for example, on macOS when the app
> window is not frontmost), the tool returns an actionable error message with suggestions,
> such as instructing the user to bring the app to the foreground and retry.
````
### 3. macOS troubleshooting note (new or existing section)
In any macOS-specific troubleshooting or known issues section:
````markdown
#### Screenshot fails on macOS when the app is not active
**Symptom**: `maui_screenshot` or `ScreenshotResultAsync` returns an error with reason
`window-not-frontmost`.
**Cause**: The macOS window server may purge the window's backing store when the app is
not the frontmost application, causing CGWindowListCreateImage to return null.
**Fix**: Bring the MAUI app window to the foreground (click it or use Cmd+Tab), then
retry the screenshot. The `ScreenshotResult.Retryable` property will be `true` to
indicate this.
> DevFlow now attempts multiple fallback capture strategies (CGWindowListCreateImage →
> CacheDisplay → DataWithPdfInsideRect) before reporting failure, so screenshots work in
> many background scenarios. The error is only returned when all strategies are exhausted.
````
> Generated by [PR Documentation Check](https://github.com/dotnet/maui-labs/actions/runs/28927372399) for issue #349 · [◷](https://github.com/search?q=repo%3Adotnet%2Fdocs-maui+is%3Aissue+%22gh-aw-workflow-call-id%3A+dotnet%2Fmaui-labs%2Fpr-docs-check%22&type=issues)
Contributor guide
Research direction
Start in docs/developer-tools/devflow/ by locating the existing ScreenshotAsync documentation and the AgentClient API reference. Check whether a maui_screenshot MCP reference or macOS troubleshooting page exists, then document ScreenshotResultAsync, ScreenshotResult properties, actionable failures, and the non-frontmost macOS behavior. Done means the affected references explain both the new API and how users should respond to retryable screenshot errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- devtools, documentation
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100