[Bug] Computer Use (zcode-cua MCP): slow on Electron apps — no tree-diff mode, no batching, silent background-typing failures

Environment · 环境

• ZCode desktop (dev.zcode.app), plugin zcode-cua 0.5.12 (computer-use MCP), model GLM-5.3-Flash
  Category · 类别: 工具调用 / MCP · Tool use / MCP | Severity: 性能 · Performance (not a crash, but a large speed gap)

• macOS (Apple Silicon), target app: Microsoft Teams (Electron)

What happened · 问题描述

A simple GUI task — "open Teams, type 'hi Eduard' in the chat with Eduard, do not press Enter" — took 1:23 end-to-end with ZCode, while a competing agent with in-process computer-use scripting completed the identical task on the same machine, same app state, in 14–32s. After optimizing my own flow I got it down to 4 tool calls (~35s), but a structural gap remains.

Root causes found · 具体原因

1. get_app_state is too heavy. A Teams window returns 921 elements (~2.5k tokens) per call, and the flow required several observations. There is no compact/diff mode. The competing bridge returns accessibility-tree diffs by default, which keeps each step cheap.
2. No action batching. Each MCP call is one UI action with a full round trip; verification is a separate call (often + screenshot). The competing bridge batches e.g. click + type + verify in a single in-process call.
3. Silent failures on Electron apps cost ~30s of recovery:
   • set_value on Teams' rich-text compose box returned target_verification_status: "mismatched" and the value was not applied — no hard error, easy to miss.
   • App-scoped background type (synthetic-focus window event) reported action_sent: true but the Electron window silently dropped the keystrokes. The receipt said "sent", nothing happened, and the failure only surfaced via a screenshot.
   • What does work reliably: open_application(activate=true) → type with an element target + strategy="event" (foreground path).
4. Redundant full-tree re-observation between actions is tempting because state_ids expire, but each dump is expensive.

Suggestions · 建议

• Add a mode: "diff" / compact option to get_app_state (elements changed since last state), and/or a filter param (e.g. editable/pressable only).
• Add a batched action API (e.g. run_sequence([{click},{type},{verify}])) so multi-step UI edits take one round trip.
• Make receipts honest about delivered vs. actually applied: fail loudly when an AX write doesn't stick, and auto-recommend the foreground path for Electron/webview targets instead of letting background dispatch silently no-op.
• One-call observe + screenshot verify for final confirmation.

Result after optimizing · 优化后

Same task re-run with: activate → one observation → type (event) → one screenshot = 4 calls, well under a minute. The remaining ~2× gap vs the in-process agent is per-call MCP round-trip overhead and tree verbosity — exactly what items 1–2 above would close.

Happy to provide session traces if useful.

Start with the zcode-cua MCP entry points for get_app_state, set_value, type, open_application, and screenshot, then reproduce the Teams flow described in the report. Compare the current behavior with the suggested diff or compact state, batched actions, and delivery verification. Done means the flow avoids redundant full-tree calls, reports Electron failures honestly, and confirms the final result.