uttrflow / uttrflow/uttrflow-swift

AI suggestion repetition filter passes "pic pic pic pic pic" and letter-spaced output

Open Beginner friendly
#538 0 comments 0 reactions 0 assignees View on GitHub
area:ai-suggestions bug good first issue P2
Dominant language
Swift
Stars
4
Forks
17
Avg merge
3h 32m
Merged PRs (30d)
277

Description

## What happens

The model's repetition filter lets two kinds of degenerate AI suggestion through to the screen. Both were seen in `uttrflow-bakeoff complete --fixtures` on `f969b41` (release, gemma3):

- `chat/group/who-is-picking/cut10`: typed `who is pic`, drawn `who is pic pic pic pic pic`
- `notes/todo/pay-electricity-bill/cut3`: typed `- p`, drawn `- p a s s p o r t`

`MLXCandidateScorer.isDegenerate` (`Sources/UttrflowLocalModel/MLXCandidateScorer.swift:577-582`) only flags a continuation of 6 or more words drawn from at most a third as many distinct words:

- ` pic pic pic pic pic` is 5 words, so the `>= 6` guard fails.
- ` a s s p o r t` is 7 words with 6 distinct, and 6 × 3 > 7.

Both fixtures are also counted as hits by the bake-off, which is #540.

## Why it matters

A word repeated five times, or letters spaced one by one, looks broken. Accepting it by habit puts garbage in a message.

## Acceptance criteria

- `isDegenerate` also flags a continuation where one word repeats 3 or more times in a row, and one made mostly of single-character "words" (for example 4 or more one-letter tokens in a row, ignoring common one-letter words such as "a" and "I" appearing alone).
- New cases in `Tests/UttrflowLocalModelTests/CompletionParsingTests.swift` for both strings above, plus a counter-example that must still pass (for example `no no no, not that one` stays, or the threshold is justified).
- `uttrflow-bakeoff complete --fixtures --only chat/group` and `--only notes/todo` no longer show these two lines.

## Where to start

- `Sources/UttrflowLocalModel/MLXCandidateScorer.swift`, `isDegenerate`
- Tests: `Tests/UttrflowLocalModelTests/CompletionParsingTests.swift`, `CompletionParsingPropertyTests.swift` (the property tests assert some continuations are *not* degenerate, so run them)
- Run `make verify`; see `CONTRIBUTING.md`.

Size: small (under 30 lines plus tests).

Contributor guide

Open the contributing guide

Research direction

Start with isDegenerate in Sources/UttrflowLocalModel/MLXCandidateScorer.swift and run the relevant tests in Tests/UttrflowLocalModelTests/CompletionParsingTests.swift and CompletionParsingPropertyTests.swift. Add coverage for the repeated-word and letter-spaced continuations, preserving a valid counter-example, then run make verify and both specified uttrflow-bakeoff fixture commands to confirm the lines no longer appear.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
ai, testing
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.