uttrflow / uttrflow/uttrflow-swift
Migrating the suggestion database lowercases accented Latin text differently from new writes
- Dominant language
- Swift
- Stars
- 4
- Forks
- 17
- Avg merge
- 3h 32m
- Merged PRs (30d)
- 277
Description
## Problem
The v1 migration populates `text_lower` with SQLite `lower(text)`, which lowercases ASCII only in this build. Normal writes and queries use Swift `lowercased()`, which also handles accented Latin letters. Upgraded entries consequently have different search keys from newly learned copies of the same text.
## Evidence
Reviewed `main` at `8d93f5b8574f3c7b8c30086d9c817bf7f5d4827d`. Reproduced against the unchanged production source in an isolated Swift package on macOS, with fake transport or temporary stores as appropriate. These checks assert the observed defect; they are not a claim that the full app test suite passed.
Against the app's SQLite layer, migrate a v1 entry whose text is `ÉCOLE report`. The resulting stored key is `École report`; Swift produces `école report`. The uppercase accented initial sorts outside the lowercase prefix range. A re-record of the identical text only updates counters and time in the UPSERT and does not repair `text_lower`.
Reproduced output: `MIGRATED_LOWERCASE ["École report"]`. Fuzzy fallback may mask some affected lookups; the exact-prefix index is nevertheless inconsistent.
## Relevant code
- [Sources/UttrflowPredictStore/Schema.swift:82](https://github.com/uttrflow/uttrflow-swift/blob/8d93f5b8574f3c7b8c30086d9c817bf7f5d4827d/Sources/UttrflowPredictStore/Schema.swift#L82)
- [Sources/UttrflowPredictStore/PredictStore.swift:177](https://github.com/uttrflow/uttrflow-swift/blob/8d93f5b8574f3c7b8c30086d9c817bf7f5d4827d/Sources/UttrflowPredictStore/PredictStore.swift#L177)
- [Sources/UttrflowPredictStore/PredictStore.swift:243](https://github.com/uttrflow/uttrflow-swift/blob/8d93f5b8574f3c7b8c30086d9c817bf7f5d4827d/Sources/UttrflowPredictStore/PredictStore.swift#L243)
## Fix and acceptance criteria
Use the same Unicode casing function for migration, writes and query keys, and repair already-migrated rows. Test a genuine v1 schema with uppercase accented Latin text, exact-prefix matching after migration, and a later identical recording.
## Existing work checked
#717 concerns canonical Unicode normalization between spellings. This is a different mismatch: ASCII-only SQLite lowercasing during migration versus Swift Unicode lowercasing.
Contributor guide
Research direction
Read Sources/UttrflowPredictStore/Schema.swift:82 and PredictStore.swift at lines 177 and 243 to trace migration, writes, and query keys. Start by reproducing the genuine v1 schema case with “ÉCOLE report,” then add coverage for migration, exact-prefix matching, and a later identical recording. Done means all three paths use consistent Unicode casing and migrated rows are repaired.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- swift
- Domain
- database
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100