uttrflow / uttrflow/uttrflow-swift

AI suggestions describe punctuated Chinese, Japanese and Hindi prose as "commands, code or queries" and give it a 32-token budget

Open Beginner friendly
#726 1 comment 0 reactions 0 assignees View on GitHub
area:ai-suggestions bug good first issue P2
Dominant language
Swift
Stars
4
Forks
17
Avg merge
3h 32m
Merged PRs (30d)
277

Description

## What happens

`Register.infer` describes where a line is being written, and the description goes straight into the suggestion model's prompt (`register.hints`, `Sources/UttrflowLocalModel/PromptBuilder.swift:74`) and sets its token budget (`register.maxTokens`, `AppleCandidateGenerator.swift:35`, `MLXCandidateScorer.swift:316`). Two of its measures assume English text:

1. **Symbol share** (`Sources/UttrflowPredict/Register.swift:223-231`) counts every visible character that is not a letter or digit as a symbol, and above 0.10 (`:33`) the text is declared "commands, code or queries rather than prose" (`:80`, `:130-133`) with a 32-token budget (`:111`). Chinese and Japanese pack a word into one or two characters, so ordinary sentence punctuation (`,`, `。`, `?`) is a much larger share of the characters; Hindi's `।` is counted the same way.
2. **Sentence case** (`:234-240`) requires the first character to be upper-case and the last to be one of `.!?`. Scripts without case never pass, and `。`, `?`, `!` and `।` are not accepted, so the person is described as writing "casually, without sentence punctuation" (`:137`).

Measured with a throwaway test on three short, fully punctuated lines per language:

| Language | symbolShare | Expected kind / hint | Actual kind / hint |
|---|---|---|---|
| English | 0.068 | line; "writes in full sentences" | line; "writes in full sentences" |
| Chinese | 0.143 | line | "command, query or line of code"; "commands, code or queries rather than prose" |
| Japanese | 0.103 | line | same as Chinese |
| Hindi | 0.111 | line | same as Chinese |

(Once symbol share is fixed, the sentence-case hint would read "writes casually, without sentence punctuation" for all three, because `sentenceCaseShare` is 0.0.)

## Why it matters

For anyone typing prose in these languages the model is told the field holds code, is given 32 tokens, and, after that is fixed, is told the writer does not punctuate. Suggestions come out shaped like commands or unpunctuated, or are cut short.

## How to reproduce

In `Tests/UttrflowPredictTests/RegisterTests.swift`:

```swift
let own = ["我们明天下午三点开会,请准时参加。", "谢谢,到时候见。", "你能把笔记发给我吗?"]
#expect(Register.symbolShare(of: own) <= Register.symbolicShare) // 0.143 today
#expect(Register.sentenceCaseShare(of: own) >= 0.5) // 0.0 today
```

## Acceptance criteria

- Sentence punctuation (Unicode general category punctuation used to end or separate clauses, including `,。?!、।`) does not count as a symbol, or the share is weighed per word rather than per character; shell and code lines keep their current classification (the existing `RegisterPropertyTests` still pass).
- A line in an uncased script counts as a sentence when it ends in sentence punctuation of any script; cased scripts keep the capital check.
- Tests cover English, Chinese, Japanese and Hindi lines, plus a shell line.

## Where to start

- `Sources/UttrflowPredict/Register.swift:222-240`.
- Tests: `Tests/UttrflowPredictTests/RegisterTests.swift`, `RegisterPropertyTests.swift`.
- Run `make verify` (export `DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer` first) and read CONTRIBUTING.md.

**Size:** S, an hour or two.

Contributor guide

Open the contributing guide

Research direction

Start in Sources/UttrflowPredict/Register.swift around lines 222–240, then read RegisterTests.swift and RegisterPropertyTests.swift. Run the supplied reproduction or the focused tests first, and use make verify with the documented DEVELOPER_DIR. Done means Chinese, Japanese, and Hindi prose receive prose classification and sentence hints, while English and shell behavior remains covered by the existing tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
macos, swift
Domain
ai, testing
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
84/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.