uttrflow / uttrflow/uttrflow-swift

Insertion finds the focused element differently from the field reader, so in a browser editor the paste check reads one word, not the field

Open
#603 0 comments 0 reactions 0 assignees View on GitHub
area:dictation bug help wanted P2
Dominant language
Swift
Stars
4
Forks
17
Avg merge
3h 32m
Merged PRs (30d)
277

Description

## What happens

The app has two ways of finding "the focused field", and they give different answers.

**The field reader** (`SurfaceProbe.focusedField`, `Sources/UttrflowContext/SurfaceProbe+System.swift:35-48`) asks the system-wide element first. It keeps that answer only if its role is a text-entry role, and otherwise asks the application:

```swift
// While a browser editor is typed into, the system names the word under the caret; the application still names the field.
if let field = systemWide, FocusedFieldSnapshot.isTextEntry(string(field, kAXRoleAttribute)) {
```

That rule came from a measured failure (`Docs/predict-reliability.md`, "Browser code editors": "a system-wide focused element that is the word under the caret while typing").

**The insertion side** (`AXAccessibilityFocus.focusedElement`, `Sources/UttrflowInput/SystemInput.swift:153-164`) never got that fix. It returns the system-wide answer whatever its role:

```swift
if let element = focusedElement(of: system) { return element }
```

Everything on the dictation insertion route reads through it:

- `focusedTextField()` (`:199-210`): whether the Accessibility write is tried, and what it writes into;
- `tail(upTo:)` (`:189-197`): what `PasteConfirmation` compares the pasted words against, every 40 ms;
- `precedingText(_:)` (`:179-186`): the check the typed completion route makes before pressing Delete.

In a browser editor where the system names the word under the caret, the paste confirmation reads that one word's value and selection. So the 24-character tail of a dictation never matches, and the result is either "Inserted — not confirmed" after 1.6 s or, when the word element has no selection, "not reported". The Accessibility write, if tried, goes to a static-text element, not to the editor.

## Why it matters

Browser editors are where people write most of their mail and documents. There, a dictation sits in "inserting" for the whole 1.6 s budget and then gets a warning it did not earn. The warning is the thing `Docs/insertion.md` says must mean something.

## How to reproduce

Needs a real browser, so `help wanted`.

1. Open a page with a rich editor (for example a `contenteditable` element holding a few paragraphs) and put the caret inside a word.
2. `uttrflow-dev insert --via paste "a sentence long enough to confirm"`, then click into the editor during the countdown.
3. Note what `uttrflow-dev` prints: "words reached the caret", "will not say" or "no sign of the words".
4. Repeat in a plain `` for comparison.

Also useful: `uttrflow-dev probe surface` in the same editor, to see which role the reader settles on.

## Acceptance criteria

- One function finds the focused field for insertion, the suggestion reader and the context engine, with the text-entry preference above. `AXAccessibilityFocus` calls it rather than keeping its own copy.
- A test (through the seam #492 proposes) shows that when the system-wide element is non-text and the application element is a text area, insertion, `tail(upTo:)` and `precedingText(_:)` all read the text area.
- `Docs/insertion.md` "Finding the focused element takes two questions" describes the role check.

Contributor guide

Open the contributing guide

Research direction

Start with SurfaceProbe.focusedField in Sources/UttrflowContext/SurfaceProbe+System.swift and AXAccessibilityFocus.focusedElement in Sources/UttrflowInput/SystemInput.swift, comparing their handling of system-wide and application elements. Use `uttrflow-dev probe surface` and the browser-editor reproduction, then add coverage through the seam proposed in #492 for insertion, `tail(upTo:)`, and `precedingText(_:)`. Done means the shared focus behavior is tested and Docs/insertion.md documents the role check.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
accessibility, desktop
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.