uttrflow / uttrflow/uttrflow-swift
Track swift-transformers' quadratic added-token regex (huggingface/swift-transformers#383) and adopt the fix when released
- Dominant language
- Swift
- Stars
- 4
- Forks
- 17
- Avg merge
- 3h 32m
- Merged PRs (30d)
- 277
Description
## What happens
The tokenizer this app uses for the suggestion model comes from swift-transformers (pinned at 1.3.4 in `Package.resolved`). Its `encode` is quadratic in the number of added tokens: about 29 ms per newline with Gemma 3, reported upstream as huggingface/swift-transformers#383. A fix is open as huggingface/swift-transformers#386, not merged or released at the time of writing.
`Docs/performance.md` (around line 407) records that nothing here patches or bumps it. #449 worked around most of the cost by tokenising the prompt's frame and lines once. What remains is the candidate scoring, which still tokenises through the slow path on every pass (#478).
## Why it matters
This is the largest remaining processor cost on the suggestion path that is not this app's code, and it is paid while the user types.
## Plan
1. Watch huggingface/swift-transformers#386 until it is merged and in a tagged release.
2. Bump `swift-transformers` in `Package.swift` and `Package.resolved`, checking whether `mlx-swift-lm` constrains its version.
3. Run `make bakeoff ARGS="gpu-memory --typing"` before and after, and confirm suggestions are identical on the scoring fixtures.
4. Update `Docs/performance.md` and remove the note that it is not bumped.
## Acceptance criteria
- The bump lands only with a released version, with before and after measurements recorded.
- No vendored or patched copy of the dependency is added in the meantime.
Contributor guide
Research direction
Monitor swift-transformers#386 until a tagged release is available, then inspect Package.swift, Package.resolved, and Docs/performance.md for the dependency and existing performance note. Run make bakeoff ARGS="gpu-memory --typing" before and after the bump, verify suggestions match the scoring fixtures, record the measurements, and update the documentation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- swift
- Domain
- build-system, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100