uttrflow / uttrflow/uttrflow-swift
Suggestion recency queries retain a new prepared statement for every learned folder count
- Dominant language
- Swift
- Stars
- 4
- Forks
- 17
- Avg merge
- 3h 32m
- Merged PRs (30d)
- 277
Description
## Problem
On main `8d93f5b8574f3c7b8c30086d9c817bf7f5d4827d`, `PredictStore.recent` creates SQL containing one placeholder per matching surface. Each additional scope changes the SQL string. `Database.statement` caches every distinct string permanently until the store is released.
- [PredictStore.recent and recentQuery](https://github.com/uttrflow/uttrflow-swift/blob/8d93f5b8574f3c7b8c30086d9c817bf7f5d4827d/Sources/UttrflowPredictStore/PredictStore.swift#L65-L95)
- [SQLite statement cache](https://github.com/uttrflow/uttrflow-swift/blob/8d93f5b8574f3c7b8c30086d9c817bf7f5d4827d/Sources/UttrflowPredictStore/SQLite.swift#L25-L74)
- [Shipping call in SuggestionCoordinator.situation](https://github.com/uttrflow/uttrflow-swift/blob/8d93f5b8574f3c7b8c30086d9c817bf7f5d4827d/Sources/Uttrflow/Suggestion/SuggestionCoordinator.swift#L556-L562)
In a long-lived store that learns new folders and runs a recency query after each, it retains the queries for 1, 2, 3, ... N surfaces. Each statement also grows with its placeholder count. The retained statement memory grows approximately quadratically. The 2,000-entry-per-surface cap does not bound this cache.
## Measured reproduction
Used the unchanged production store with `:memory:` on macOS, one entry per scope. For each new scope, record `echo sample`, then call `recent(in: surface, limit: 6)`. All surfaces share a bundle, role and locator and differ only by `/project/dir-N` scope. Compare with the same records without calling `recent`.
After each awaited operation, inspect the otherwise idle store's cached statement handles and sum `sqlite3_stmt_status(statement, SQLITE_STMTSTATUS_MEMUSED, 0)`:
| Scopes learned and queried during this store lifetime | Cached statements | Statement bytes |
| --- | --- | --- |
| 100 | 109 | 2,302,848 |
| 250 | 259 | 10,453,456 |
| 500 | 509 | 37,090,304 |
| 1,000 | 1,009 | 141,340,960 |
Control: recording the same 100–1,000 scopes without recency queries stays at **8 statements / 55,088 bytes**. Both runs hold exactly 1,000 entries at completion. The 58 existing suggestion-store tests passed.
These numbers measure SQLite prepared-statement allocations, not total app RSS. They demonstrate growth while new scopes are learned in one store lifetime; merely reopening a pre-existing 1,000-scope database does not compile all historical query shapes. Repeating a query at an unchanged surface count reuses its statement.
## Acceptance criteria
- Recency reads use a stable SQL shape (for example, select matching surface IDs inside SQL), or enforce a bounded statement cache with proper finalization on eviction.
- Preserve current-document precedence, text deduplication, superseded/self-sourced filtering and the requested result limit.
- Add a regression that interleaves learning new scopes with recency reads and verifies statement retention remains bounded; compare results with current recency semantics.
Priority: P2 (retained memory on the suggestion path). Separate from #775, which concerns persistent succession rows rather than prepared statements.
Contributor guide
Research direction
Read PredictStore.recent and recentQuery in Sources/UttrflowPredictStore/PredictStore.swift, then inspect the statement caching in Sources/UttrflowPredictStore/SQLite.swift and the call from SuggestionCoordinator.situation. Use the existing suggestion-store tests as a starting point and add a regression that learns scopes between recency reads, verifies retention stays bounded, and preserves current result semantics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sqlite, swift
- Domain
- databases, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100