[Swift runtime] Pipelined prefix reuse skips the last sampled token
- Dominant language
- Swift
- Stars
- 2.1k
- Forks
- 202
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 55
Description
### Model name
Qwen3.5 0.8B GDN S=1, Qwen3.8 27B GDN S=1, and dynamic Qwen3 0.6B (public bundle revisions and full file hashes in the linked evidence).
### What happens
On official `27a66f90e7f3fd9b83a6acb7bcb0a4a5ff71fd60`, a completed 24-token generation from a 24-token prompt leaves `processedTokenCount == 47`, while history contains 48 tokens. Extending that history to 68 input tokens reports a prefix hit of 48 and skips the last sampled token. After another 24 outputs, the processed count is 90; a full replay gives the expected 91.
The 27B greedy continuation differs from full replay (“The Eiffel Tower…” vs “One famous landmark in Paris…”). The count discrepancy also occurs on the dynamic control before #227, so I am reporting this separately from #212/#227. Full reset/replay is repeatable.
A four-line patch limits reusable history to `processedTokenCount`. The same three fixtures then pass count and token parity; all seven dynamic generation phases remain token-identical. Hybrid partial reset remains explicitly rejected. [Review copy of patch and opt-in regression test](https://github.com/john-rocky/coreai-models/pull/1). Related test-coverage tracking: #88. This is not a claim covering cancellation/early-EOS or every multi-turn case.
### Command run
The following creates an isolated reproduction checkout and reverses only the patch, retaining the consumer test package. Run with a platform-compatible macOS bundle and your shared GPU lock held:
```sh
REPRO_DIR=$(mktemp -d)
git clone --branch fix/pipelined-pending-token https://github.com/john-rocky/coreai-models.git "$REPRO_DIR"
cd "$REPRO_DIR"
git checkout --detach a40d26eaeb7dd5492c24edc178929e8e40daa184
git revert --no-commit 48281854f9834cdcc61b9cb320a960835eb1e8f2
export DEVELOPER_DIR=/Applications/Xcode-27.0.0-Beta.5.app
export COREAI_CONTINUATION_BUNDLE=/absolute/path/to/macos/bundle
unset COREAI_CHUNK_THRESHOLD
swift test --package-path validation/2026-09-05-coreai/consumer -c release --force-resolved-versions --filter PipelinedContinuationTests
```
The regression covers budgets 1 and 4, successive extensions, exact continuation and full replay. It fails on the unpatched source and passes with the patch on the 0.8B S=1 and dynamic fixtures. Separate logs include 24-token text generation on the 27B fixture.
### macOS / iOS target
Mac Studio `Mac16,9`, Apple M4 Max, 128 GiB; macOS 27.0 `26A5416b`. No iPhone run.
### Xcode version
Xcode 27 beta 5 `27A5237l`; macOS SDK 27.0 `26A5406c`; Swift 6.4.
### Full error output and evidence
[Results, exact revisions, commands and downloadable logs](https://github.com/john-rocky/coreai-models/blob/a40d26eaeb7dd5492c24edc178929e8e40daa184/validation/2026-09-05-coreai/RESULTS.md). The archive is 109,859 bytes with SHA-256 `a63d2700eed17c0e0b7c6b9242b40249cce792afbc3856e1b5df2b1b2a7a1913`.
The new regression test records `processedTokenCount` and prefix-hit expectation failures on the unpatched engine. For example, the third turn processes 16 tokens where 18 are expected, and the 0.8B output also diverges from replay in that case. The corrected engine passes both cases.
I am using an issue with a forked review copy in accordance with the repository's current contribution policy.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.