uttrflow / uttrflow/uttrflow-swift

AI suggestions split a word typed mid-way ("func subtr" → "func subtr act(a: Int"): 17 of 35 wrong answers in the fixtures

Open
#539 0 comments 0 reactions 0 assignees View on GitHub
area:ai-suggestions bug P2
Dominant language
Swift
Stars
4
Forks
17
Avg merge
3h 32m
Merged PRs (30d)
277

Description

## What happens

When the typed line ends inside a word, the model's pass may treat the fragment as a finished word and start a new one after a space. The AI suggestion then splits the word the person was typing.

`uttrflow-bakeoff complete --fixtures --model gemma3 --raw` on `f969b41` (release, 1,154 fixtures): 982 lines drawn, 35 wrong. **51** drawn lines put a space or hyphen directly after a typed fragment ending in a letter or digit, and **17 of the 35 wrong answers** are that shape:

| Fixture | Typed | Drawn |
|---|---|---|
| `code/swift/func-subtracta-int/cut10` | `func subtr` | `func subtr act(a: Int` |
| `code/typescript/export-function-subtracta/cut21` | `export function subtr` | `export function subtr act(a: number` |
| `notes/groceries/toothpaste/cut5` | `tooth` | `tooth paste` |
| `notes/groceries/paneer/cut3` | `pan` | `pan cakes` |
| `notes/groceries/onions/cut3` | `oni` | `oni onions` |
| `notes/groceries/yoghurt/cut2` | `yo` | `yo apples` |
| `chat/hinglish/main-nikal-raha/cut7` | `main ni` | `main ni pata` |
| `chat/hinglish/main-nikal-raha/cut13` | `main nikal ra` | `main nikal ra hoon` |
| `terminal/node/npx-eslint-src/cut7` | `npx esl` | `npx esl-watch` |

Several more are scored as hits only because their fixture accepts any continuation (`notes/release/the-beta-group/cut6` `The be` → `The be-release…`, `notes/report/our-next-milestone/cut13` `Our next mile` → `Our next mile is focused…`, `chat/work/standup-notes-are/cut10` `Standup no` → `Standup no problem.`).

**Why.** `TokenHealing` holds the model to the typed fragment, but once the fragment is written out exactly (`owed` empty), `Vocabulary.allowed(owing:wordComplete:)` admits any token with a visible character, including one that *opens with a space* (`Sources/UttrflowLocalModel/TokenHealing.swift:34-36`). `took` then frees the model on it (`:80-82`). Only `wordComplete` (a typed trailing space) forces the space. Nothing prefers lengthening the word when the person stopped mid-word.

This is the generation path. #58, #338 and #477 are about the *scorer* judging remembered lines cut mid-token, and do not cover it.

## Why it matters

A suggestion that breaks the word being typed reads as a typo-maker. Accepting it inserts `subtr act` into code. It hits short list items and Hinglish chat hardest.

## Acceptance criteria

- When the typed line ends in a letter or digit with no trailing space, the first free token after the fragment prefers continuing the word: for example, a space-led token is allowed only if the model's probability for word-continuing tokens is below a measured threshold, or a pass that splits the fragment into a non-word is dropped.
- Re-run `uttrflow-bakeoff complete --fixtures`: the split-fragment count and the wrong answers above are reported before and after, with no loss on fixtures where the fragment really is a whole word (`robust/short-command/ls/cut2` `ls` → `ls -la`, `chat/yes`).
- Unit tests in `Tests/UttrflowLocalModelTests/TokenHealingTests.swift` for the new rule.

Contributor guide

Open the contributing guide

Research direction

Start with Sources/UttrflowLocalModel/TokenHealing.swift, especially the Vocabulary.allowed and took paths at the cited lines, to understand how a typed fragment becomes unconstrained. Read Tests/UttrflowLocalModelTests/TokenHealingTests.swift and run the fixture bakeoff command. Done means the new unit tests pass and the reported split-fragment cases improve without regressing whole-word continuations.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
ai, machine-learning, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.