uttrflow / uttrflow/uttrflow-swift

Track swift-transformers' quadratic added-token regex (huggingface/swift-transformers#383) and adopt the fix when released

Open
#489 0 comments 0 reactions 0 assignees View on GitHub
area:ai-suggestions enhancement P2 performance
Dominant language
Swift
Stars
4
Forks
17
Avg merge
3h 32m
Merged PRs (30d)
277

Description

## What happens

The tokenizer this app uses for the suggestion model comes from swift-transformers (pinned at 1.3.4 in `Package.resolved`). Its `encode` is quadratic in the number of added tokens: about 29 ms per newline with Gemma 3, reported upstream as huggingface/swift-transformers#383. A fix is open as huggingface/swift-transformers#386, not merged or released at the time of writing.

`Docs/performance.md` (around line 407) records that nothing here patches or bumps it. #449 worked around most of the cost by tokenising the prompt's frame and lines once. What remains is the candidate scoring, which still tokenises through the slow path on every pass (#478).

## Why it matters

This is the largest remaining processor cost on the suggestion path that is not this app's code, and it is paid while the user types.

## Plan

1. Watch huggingface/swift-transformers#386 until it is merged and in a tagged release.
2. Bump `swift-transformers` in `Package.swift` and `Package.resolved`, checking whether `mlx-swift-lm` constrains its version.
3. Run `make bakeoff ARGS="gpu-memory --typing"` before and after, and confirm suggestions are identical on the scoring fixtures.
4. Update `Docs/performance.md` and remove the note that it is not bumped.

## Acceptance criteria

- The bump lands only with a released version, with before and after measurements recorded.
- No vendored or patched copy of the dependency is added in the meantime.

Contributor guide

Open the contributing guide

Research direction

Monitor swift-transformers#386 until a tagged release is available, then inspect Package.swift, Package.resolved, and Docs/performance.md for the dependency and existing performance note. Run make bakeoff ARGS="gpu-memory --typing" before and after the bump, verify suggestions match the scoring fixtures, record the measurements, and update the documentation.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
build-system, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.