Computer Use cannot show the model what select_text actually selected
- Dominant language
- TypeScript
- Stars
- 5.4k
- Forks
- 502
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 706
Description
### Problem
`maka_computer` already accepts `select_text`. The maka-cu executor reports the selected string on the snapshot as `selectedText: { text, truncated }`, and the host backend copies it onto the observation object. The shared `CuObservation` type does not declare that field, and `renderObservationForModel` never writes it. After a successful select, the model only sees `ok` plus the element tree. `element.selected` is AX control selection (checkbox/row), not the window text range. `select_text` also uses screenshot policy `never`, so there is no picture of the highlight either. The next turn cannot confirm what was selected.
### Desired outcome
A model that just called `select_text` can read the selected string from the fresh observation header, truncated the same way element `value` is truncated. If the focused control is a secure field, the selected string is withheld rather than printed. Observations with no selection do not mention the field.
### Alternatives or workarounds
Asking the model to re-observe and guess from element values. That is slower and still does not report a range that spans multiple elements.
Contributor guide
Assessment
This issue has not been assessed yet.