uttrflow / uttrflow/uttrflow-swift

VoiceOver is never told an AI suggestion is on screen, yet Tab and Escape are taken while it is

Open
#532 0 comments 0 reactions 0 assignees View on GitHub
accessibility area:ai-suggestions bug P1
Dominant language
Swift
Stars
4
Forks
17
Avg merge
3h 32m
Merged PRs (30d)
277

Description

## What happens

An AI suggestion is drawn in a floating panel and never announced, so a VoiceOver user does not know one is on screen, yet the accept key is armed while it is.

- `SuggestionView` (`Sources/Uttrflow/Suggestion/SuggestionView.swift:23-24`) gives the surface a combined element and an `accessibilityLabel` built by `SuggestionPresentation.accessibilityLabel` (`Sources/UttrflowUX/SuggestionPresentation.swift:141-149`), e.g. "Suggestion: … Tab to accept."
- Nothing ever reads that label out. There is no `NSAccessibility.post(…, .announcementRequested, …)` anywhere under `Sources/Uttrflow/Suggestion/` (a search for `NSAccessibility.post` in `Sources/` finds none at all).
- The panel is a non-activating `NSPanel` at `.statusBar` level with `ignoresMouseEvents = true` (`SuggestionPanelController.swift:155-172`). VoiceOver's cursor stays in the other application's field and cannot reach it.
- Meanwhile `SuggestionCoordinator.draw` arms the tap (`SuggestionCoordinator.swift:594-608`), and `KeyRouting.decision` takes Tab (or the application's accept key) and Escape (`Sources/UttrflowPredict/KeyRouting.swift:85-94`).

So for a VoiceOver user, Tab in a web form or a mail field sometimes moves focus and sometimes silently types text they never heard, and Escape sometimes does nothing. Nothing checks `NSWorkspace.shared.isVoiceOverEnabled` either.

## Why it matters

A key that changes meaning based on something invisible breaks keyboard and screen-reader use. A blind user may send a message containing a completion they never heard. This is the core accept flow, so P1.

## How to reproduce

Code reading (above). With VoiceOver on and AI suggestions on, type a remembered line in a text field. VoiceOver says nothing about the grey text. Press Tab: the completion is inserted instead of focus moving.

## Acceptance criteria

- When a suggestion first appears for a line (not on every redraw), its text and accept key are announced with `NSAccessibility.post(element:notification:.announcementRequested, userInfo:)` at a low priority, so typing echo is not interrupted. The label that already exists is the text.
- Or, as a product decision, AI suggestions arm no keys while VoiceOver is running unless the user opts in. Either way a VoiceOver user is never given an accept key they were not told about.
- A test in `Tests/UttrflowUXTests` covers the announcement text for a single completion, a replacement and a list.

Contributor guide

Open the contributing guide

Research direction

Start with SuggestionView.swift, SuggestionPresentation.swift, SuggestionPanelController.swift, SuggestionCoordinator.draw, and KeyRouting.decision to trace the suggestion lifecycle and existing label. Then inspect Tests/UttrflowUXTests and determine how to cover a single completion, replacement, and list; done means VoiceOver users are informed before the accept key is active, without repeated announcements.

Written by the indexing model from the issue text.

Assessment

Tech stack
macos, swift
Domain
accessibility, desktop
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.