Computer Use loses Shift through Tart or Screen Sharing into a macOS guest

Summary

Computer Use type_text and shifted press_key input lose Shift when the target is a running macOS guest through either Tart’s built-in window or Apple Screen Sharing. Ordinary characters reach the guest, but uppercase letters and shifted punctuation arrive as their unshifted keys.

Environment

• ChatGPT/Codex desktop: 26.825.51511 (7377), Apple silicon
• Computer Use plugin: 1.0.1000633
• @oai/sky: 0.6.24-premerge-pr-1369830-395ab116910c
• host: macOS 26.5.2 (25F84), Mac16,10, U.S./ABC input sources
• Tart: official 2.32.1 app, window targeted as com.github.cirruslabs.tart
• guest: macOS 26.6.2 (25G83), U.S. keyboard layout

Minimal reproduction

1. Run a macOS guest in Tart and focus a blank TextEdit document inside the guest.
2. Target the Tart app with the documented macOS Computer Use API:

await sky.type_text({
  app: "com.github.cirruslabs.tart",
  text: "abc ABC @ ? _ + : / ' \"",
});

3. Read the visible guest field.

Expected:

abc ABC @ ? _ + : / ' "

Observed:

abc abc 2 / - = ; / ' '

Sending individual shifted chords such as shift+a, shift+2, shift+slash, shift+minus, shift+equal, shift+semicolon, and shift+apostrophe through press_key also fails to produce the requested shifted characters. Sending type_text one character per awaited call does not change the result.

Controls

• The same type_text literal matrix targeted directly at host TextEdit preserves uppercase and shifted punctuation (apart from TextEdit's normal smart-quote substitution), so the text-to-key mapping works in an ordinary host text field.
• Host and guest layouts are both U.S.; neither has a custom modifier mapping.
• The guest's built-in Accessibility Keyboard, enabled visibly in System Settings and clicked through Computer Use, inserts @, uppercase letters, and ? correctly into the same guest TextEdit field. A representative Qa@example.com and Can qa type this? were entered without shell, clipboard, or backend input. The setting was then turned off.
• A neutral TextEdit field fails, so this is not application-specific to the product under test.

The original run isolated the failure from the guest layout and target application; the additional Screen Sharing reproduction below further narrows it to synthetic input crossing a remote-display or virtualization forwarding boundary. I have not tried a custom event injector or private Computer Use build.

Additional reproduction via Screen Sharing (2026-09-03)

The failure also reproduces without targeting Tart's built-in window:

1. Start a disposable guest with tart run --vnc and connect to its exact address with Apple Screen Sharing.
2. Focus a blank TextEdit document in the guest.
3. Target com.apple.ScreenSharing with Computer Use.
4. Send type_text("@"); TextEdit visibly receives 2.
5. On a new line, send press_key("shift+2"); TextEdit again visibly receives 2.

The unrelated Tart guest already running on the host was not targeted or disturbed. The disposable reproduction guest was stopped and deleted afterward.

This broadens the isolation: the defect is not specific to Tart's built-in window. It appears when Computer Use synthetic keyboard events cross a remote-display or Virtualization.framework forwarding boundary. Direct host TextEdit remains the passing control, while the guest Accessibility Keyboard remains the passing in-guest control.

Related reports and API contract

• openai/tart#1167 reports modifier-based special symbols being mistranslated inside a macOS VM: https://github.com/openai/tart/issues/1167
• VirtualBuddy discussion #741 reports Shift, Option, and Command being lost when TeamViewer-generated input is forwarded into a Virtualization.framework guest: https://github.com/insidegui/VirtualBuddy/discussions/741
• UTM issue #7118 reports other modifier combinations being partially dropped at a VM boundary: https://github.com/utmapp/UTM/issues/7118
• Apple documents that shifted characters require explicit modifier key-down/key-up sequencing and modifier flags: https://developer.apple.com/documentation/coregraphics/cgevent/init%28keyboardeventsource%3Avirtualkey%3Akeydown%3A%29 and https://developer.apple.com/documentation/coregraphics/cgeventflags

Expected behavior

type_text and documented shifted press_key chords targeted at a macOS guest through Tart or Screen Sharing should deliver uppercase and shifted punctuation to the focused guest field, matching ordinary host typing.

Reproduce the failure with sky.type_text and press_key targeting com.github.cirruslabs.tart and com.apple.ScreenSharing, comparing the host TextEdit control with a macOS guest. Trace the synthetic keyboard event path across the remote-display or virtualization boundary; done when shifted letters and punctuation arrive correctly in the guest for both APIs.