uttrflow / uttrflow/uttrflow-swift

Migrating the suggestion database lowercases accented Latin text differently from new writes

Open
#776 0 comments 0 reactions 0 assignees View on GitHub
area:ai-suggestions bug P2
Dominant language
Swift
Stars
4
Forks
17
Avg merge
3h 32m
Merged PRs (30d)
277

Description

## Problem

The v1 migration populates `text_lower` with SQLite `lower(text)`, which lowercases ASCII only in this build. Normal writes and queries use Swift `lowercased()`, which also handles accented Latin letters. Upgraded entries consequently have different search keys from newly learned copies of the same text.

## Evidence

Reviewed `main` at `8d93f5b8574f3c7b8c30086d9c817bf7f5d4827d`. Reproduced against the unchanged production source in an isolated Swift package on macOS, with fake transport or temporary stores as appropriate. These checks assert the observed defect; they are not a claim that the full app test suite passed.

Against the app's SQLite layer, migrate a v1 entry whose text is `ÉCOLE report`. The resulting stored key is `École report`; Swift produces `école report`. The uppercase accented initial sorts outside the lowercase prefix range. A re-record of the identical text only updates counters and time in the UPSERT and does not repair `text_lower`.

Reproduced output: `MIGRATED_LOWERCASE ["École report"]`. Fuzzy fallback may mask some affected lookups; the exact-prefix index is nevertheless inconsistent.

## Relevant code

- [Sources/UttrflowPredictStore/Schema.swift:82](https://github.com/uttrflow/uttrflow-swift/blob/8d93f5b8574f3c7b8c30086d9c817bf7f5d4827d/Sources/UttrflowPredictStore/Schema.swift#L82)
- [Sources/UttrflowPredictStore/PredictStore.swift:177](https://github.com/uttrflow/uttrflow-swift/blob/8d93f5b8574f3c7b8c30086d9c817bf7f5d4827d/Sources/UttrflowPredictStore/PredictStore.swift#L177)
- [Sources/UttrflowPredictStore/PredictStore.swift:243](https://github.com/uttrflow/uttrflow-swift/blob/8d93f5b8574f3c7b8c30086d9c817bf7f5d4827d/Sources/UttrflowPredictStore/PredictStore.swift#L243)

## Fix and acceptance criteria

Use the same Unicode casing function for migration, writes and query keys, and repair already-migrated rows. Test a genuine v1 schema with uppercase accented Latin text, exact-prefix matching after migration, and a later identical recording.

## Existing work checked

#717 concerns canonical Unicode normalization between spellings. This is a different mismatch: ASCII-only SQLite lowercasing during migration versus Swift Unicode lowercasing.

Contributor guide

Open the contributing guide

Research direction

Read Sources/UttrflowPredictStore/Schema.swift:82 and PredictStore.swift at lines 177 and 243 to trace migration, writes, and query keys. Start by reproducing the genuine v1 schema case with “ÉCOLE report,” then add coverage for migration, exact-prefix matching, and a later identical recording. Done means all three paths use consistent Unicode casing and migrated rows are repaired.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
database
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.