uttrflow / uttrflow/uttrflow-swift

The app target holds 3% of the tests and every bug found this week: a four-layer plan

Open
#145 1 comment 0 reactions 0 assignees View on GitHub
area:build area:general confirmed enhancement P1
Dominant language
Swift
Stars
4
Forks
17
Avg merge
3h 32m
Merged PRs (30d)
277

Description

**P1 — the app target holds 3% of the tests and 100% of the bugs found this week.**

### The gap, measured

```
all tests: 4,490
app-target tests: 144 (3%)
UI tests: 0
Xcode project: none (SwiftPM only, so no UI test bundle can exist today)
```

Every defect found this week — #123, #132, #140, #141, #142, #143 — lives in the app target
or at the system boundary it wraps. None was caught by the 4,346 tests outside it.

This is the architecture working as intended, not a failure of it: *"All the deciding lives in
Swift Package Manager modules; the app target holds windows, menu items and SwiftUI views, and
no judgement at all."* The logic is exhaustively covered. What is left over is **wiring and
lifecycle**, and that has 16 test files, most of them cosmetic — icons, palettes, ring geometry,
thumbnails.

### Why "add UI tests" is the wrong first answer

XCUITest is the Playwright equivalent on Apple platforms, and it would have caught **one** of the
six:

| issue | caught by XCUITest? | what actually catches it |
|---|---|---|
| #123 Check Now inert | yes | a wiring test (cheaper, milliseconds) |
| #132 arrow keys open the mic | no | a unit test on the recogniser |
| #140 crash on quit | no | a soak run of hours |
| #141 tap never re-enabled | no | a seam that lets the disable be injected |
| #142 ⇧⌘V pastes | no | a helper process holding the hotkey |
| #143 immortal controller | no | a deinit assertion |

They all sit in the seam between well-tested logic and the system boundary. Driving the UI tests
the wrong side of that seam.

### Four layers, cheapest first

**1. Wiring tests — done, see the PR that closes this checkbox's first line.**
Headless, in the app target, milliseconds. `MainIntentWiringTests` was the pattern; it now
extends to every `SettingsChange`, with an exhaustive switch so a new case cannot be added
without saying what it does. Extend the same shape to `MainIntent` and `MenuBarIntent`.

**2. A soak harness.** Launch, drive synthetic activity for hours, assert object counts and heap
do not grow, then quit and assert a clean exit. This is the only thing that finds #140, and it is
what that issue currently asks somebody to do by hand. Needs to run with suggestions on, since
the 12.8-hour crash had them enabled.

**3. Seams at the system boundary.** `TapDisableWindow` already proves the shape: the *rule* is a
tested value, the tap is a thin shell. `SystemKeyboard` has no such rule, which is exactly why
#141 is untestable today. Same for the Carbon registration path in #142.

**4. XCUITest, last.** Real value for what nothing checks today: the app launches, the windows
appear, the panels open, Settings renders every pane. Not for the bug classes above.

### On XCUITest specifically, since it will be asked

Worth doing, but shaped as a **pre-release gate, not a per-PR check**:

- SwiftPM cannot host a UI test bundle. It needs a generated `.xcodeproj` (XcodeGen or Tuist,
config checked in, output not). `xcodebuild` is already used for the app bundle and the
bake-off, so this is not a new dependency.
- The accessibility tree is already in good shape — every Settings control has a real role and
title, which is the expensive half of adopting XCUITest and is already paid for.
- **TCC is the wall.** XCUITest cannot grant Accessibility or Microphone, so anything touching
the tap or dictation needs a dedicated Mac with TCC pre-seeded. Not possible on hosted runners.
- `AGENTS.md` calls CI minutes the scarce resource and UI tests are slow, which is the second
reason these belong on release candidates rather than every pull request.
- **Run the same flows repeatedly.** This matters more than it sounds: #140 only appears after
hours of accumulation, and a suite that runs each flow once would never see it. Repetition is
what turns a UI suite into a detector for this project's actual failure mode — which merges
layer 4 into layer 2.

### Definition of done

Not "we have UI tests". A release is gated on: every control reaching something (1), a soak run
that ends in a clean quit with flat object counts (2), and a smoke pass that the app launches and
every window opens (4). (3) is what makes (2) and (4) able to fail for a reason you can act on.

Contributor guide

Open the contributing guide

Research direction

Read AGENTS.md and the existing MainIntentWiringTests in the app target first, then trace the four proposed layers and their mentioned boundaries, including SwiftPM and the generated Xcode project requirement. Done means controls reach their actions, a soak run quits cleanly with flat object counts, and a smoke pass launches the app and opens every window.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
build-system, desktop, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.