uttrflow / uttrflow/uttrflow-swift
The Accessibility write is checked with one immediate read, so an app that applies it a moment later can get the words twice
- Dominant language
- Swift
- Stars
- 4
- Forks
- 17
- Avg merge
- 3h 32m
- Merged PRs (30d)
- 277
Description
## What happens
`AXTextField.replaceSelection(with:)` (`Sources/UttrflowInput/SystemInput.swift:218-234`) does four things:
1. copies the field's whole `kAXValueAttribute`;
2. sets `kAXSelectedTextAttribute`;
3. copies the whole value again **straight away**;
4. throws `"the field accepted the text and did not change"` if the two copies are equal.
That throw sends the coordinator on to the next strategy (`TextInsertionCoordinator.swift:30-37`), and for a dictation the next strategy is the paste. So the check assumes the target has applied the write, and republished its value, before the `AXUIElementSetAttributeValue` call returns.
Web engines that run page content in a separate process do not have to work that way. There the set is forwarded to the content process, and the value the Accessibility tree reports is refreshed afterwards. An app like that would see:
- the write succeeds, but the immediate read still shows the old value, so the engine throws;
- the paste then inserts the same words;
- the forwarded write lands too, so the words appear **twice**.
`Docs/insertion.md` ("The Accessibility write that changes nothing") describes web-runtime apps whose write "does nothing at all". The observation was made with the same one immediate read, so it cannot tell "never applied" from "applied a moment later".
The completion route has the same check in its path. `replaceSelection(replacing:with:)` (`SystemInput.swift:237-249`) selects backwards, writes, and on the throw moves the caret back and falls through to `TypedTextInsertionEngine`. That route presses Delete over the replaced characters and types the completion. If the Accessibility write lands late as well, the line is mangled.
Two smaller effects of comparing the whole value:
- Dictating the same words that are selected (select "okay", say "okay") is a real write that leaves the value unchanged, so it is reported as a refusal and pasted a second time.
- Each insertion copies the entire document twice across processes. In a long document that is two full string transfers, under a 2 s messaging timeout (`SystemInput.swift:134`).
## Why it matters
Doubled words are among the most visible failures a dictation tool can have, and the cause would be hard to see from the user's side. The Diagnostics route just says "paste".
## How to reproduce
Needs real apps, so `help wanted`.
1. `swift build --product uttrflow-dev`, and grant the binary Accessibility.
2. For each browser you have, and for one desktop app built on a bundled browser engine: open a page with a `` and a `contenteditable` element.
3. `uttrflow-dev insert --via accessibility "one two three"`, click into the field during the countdown. Note whether the command reports a refusal and whether the text appears, then or a moment later.
4. Run it again without `--via` (the full route) and check whether the words appear once or twice.
Report the app, the field kind, and both outcomes.
## Acceptance criteria
- A write reported as successful is not declared "did not change" on one immediate read. Either poll briefly under a small elapsed-time budget (the pattern `PasteConfirmation` uses), or check the text before the caret instead of the whole value.
- If a late-applied write is still possible when the route falls through to paste, the fall-through first re-reads, so the words cannot land twice.
- Replacing a selection with identical text is not reported as a refusal.
- Tests use a fake field that applies the write on its second or third read (#492 asks for the seam this needs). They cover: one insertion only; the identical-text case; and the completion route not deleting characters after a late write.
Contributor guide
Research direction
Start with replaceSelection(with:) and replaceSelection(replacing:with:) in Sources/UttrflowInput/SystemInput.swift, then trace the fallback in TextInsertionCoordinator.swift and the PasteConfirmation polling pattern. Use the test seam requested in #492 to model a field that updates on its second or third read. Done means late writes, identical-text replacements, and completion replacements are covered without duplicate insertion or deletion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- macos, swift
- Domain
- accessibility, desktop, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100