[macOS] Computer Use get_app_state screenshot is clipped to the display; window portions beyond the display edge come back white (blank captures with AeroSpace/yabai and any half-offscreen window)

Summary

SkyComputerUseService (com.openai.sky.CUAService 26.831.1000926, shipped with ChatGPT 26.901.51231 and used by Codex CLI 0.153.4 through the unified-computer-use plugin) captures app screenshots with a display-bound ScreenCaptureKit filter. Anything outside the display rectangle is rendered as opaque white. Tiling window managers such as AeroSpace hide inactive-workspace windows by parking them at a screen corner with one pixel left on the display, so every such window returns a solid white screenshot even though macOS can still capture it in full. The accessibility tree still works, so the agent acts blind.

Switching the single-window capture to SCContentFilter(desktopIndependentWindow:) (already linked in the service binary) returns the full window for every app tested except Final Cut Pro, which itself stops drawing while offscreen.

Environment

Reproduction (no window manager needed)

1. Open TextEdit with any document. Float the window and place it so half of it hangs past the right edge of the display, e.g. AppleScript set bounds of window 1 to {2940, 200, 3940, 900} on a 3440-wide display.
2. From Codex CLI: const app = await cua.getApp("com.apple.TextEdit"), then await app.getAXStateAndScreenshot().
3. Result: a 1000×700 JPEG whose left half shows the document and whose right half is pure white (every pixel 255).
4. Control: screencapture -l <windowID> or SCScreenshotManager.captureImage with SCContentFilter(desktopIndependentWindow:) on the same window ID returns the full window including the off-display half.

Measurements from the affected machine (September 9, 2026)

AeroSpace parks inactive windows at the bottom-right corner minus one pixel. CoreGraphics reports them as on-screen:

Captures of the same window IDs, without moving anything:

Occlusion check: with a floating TextEdit window placed on top of a visible Superset window, the Computer Use screenshot of Superset shows Superset content under the overlap, not TextEdit. So the current filter already excludes other windows; it only clips to the display rectangle.

What the binary shows

strings on SkyComputerUseService contains initWithDisplay:includingWindows:, initWithDesktopIndependentWindow:, captureImageWithFilter:configuration:completionHandler:, setSourceRect:, setShouldBeOpaque:, cannotClickOffscreenElement, and Capture Accelerated Window Screenshot. The observed clipping matches the display-including-windows filter with a source rect equal to the window frame.

Requested fix

Use SCContentFilter(desktopIndependentWindow:) for the per-window screenshot in get_app_state, with shouldBeOpaque and ignoreShadowsSingleWindow as today. Keep the display-bound path only as a fallback when the window is not in shareable content. This restores captures for windows parked by AeroSpace, yabai, and Amethyst, and for any window the user has dragged partly off-screen.

Known limit that the fix cannot cover: Final Cut Pro stops rendering when its window is almost entirely offscreen (all four capture methods return blank while it is parked and full content once visible), so it will still need to be on a display.

Reproduction probe

Swift probe used for the comparison rows above (needs Screen Recording permission; swiftc -O -o sckprobe sckprobe.swift, then ./sckprobe <windowID> out.png independent|display):

import Foundation
import ScreenCaptureKit
import CoreGraphics
import ImageIO
import UniformTypeIdentifiers

// Usage: sckprobe <windowID> <out.png> [display|independent]
let args = CommandLine.arguments
let wid = CGWindowID(UInt32(args[1])!)
let out = URL(fileURLWithPath: args[2])
let mode = args.count > 3 ? args[3] : "independent"

func save(_ img: CGImage, _ url: URL) {
    let dest = CGImageDestinationCreateWithURL(url as CFURL, UTType.png.identifier as CFString, 1, nil)!
    CGImageDestinationAddImage(dest, img, nil); CGImageDestinationFinalize(dest)
}

_ = CGMainDisplayID() // init CGS connection
let sem = DispatchSemaphore(value: 0)
Task {
    do {
        let content = try await SCShareableContent.excludingDesktopWindows(false, onScreenWindowsOnly: false)
        guard let win = content.windows.first(where: { $0.windowID == wid }) else {
            print("window \(wid) not in shareable content (count=\(content.windows.count))"); sem.signal(); return }
        print("window:", win.windowID, win.owningApplication?.applicationName ?? "?", "frame:", win.frame, "isOnScreen:", win.isOnScreen, "isActive:", win.isActive)
        let filter: SCContentFilter
        if mode == "display" {
            let disp = content.displays.first(where: { $0.frame.intersects(win.frame) }) ?? content.displays[0]
            filter = SCContentFilter(display: disp, including: [win])
            print("mode: display-bound, display frame:", disp.frame)
        } else {
            filter = SCContentFilter(desktopIndependentWindow: win)
            print("mode: desktopIndependentWindow, window frame:", win.frame)
        }
        let cfg = SCStreamConfiguration()
        if mode == "display" {
            cfg.sourceRect = CGRect(x: win.frame.minX - filter.contentRect.minX, y: win.frame.minY - filter.contentRect.minY, width: win.frame.width, height: win.frame.height)
            cfg.width = Int(win.frame.width); cfg.height = Int(win.frame.height)
        } else {
            cfg.width = Int(win.frame.width) * 2; cfg.height = Int(win.frame.height) * 2
        }
        cfg.showsCursor = false; cfg.ignoreShadowsSingleWindow = true; cfg.shouldBeOpaque = true
        let img = try await SCScreenshotManager.captureImage(contentFilter: filter, configuration: cfg)
        print("captured:", img.width, "x", img.height)
        save(img, out)
    } catch { print("ERROR:", error) }
    sem.signal()
}
sem.wait()

Happy to run further comparisons on this machine if useful. Cross-reference: #38348 is another ScreenCaptureKit-side capture issue in the same service.

Start with the get_app_state screenshot path in SkyComputerUseService and compare it with the provided sckprobe.swift modes. Verify the desktopIndependentWindow capture on a partially off-screen window, then confirm completion with full captures for parked and half-offscreen windows while preserving the documented Final Cut Pro limitation.