uttrflow / uttrflow/uttrflow-swift
CRLF model output is flattened by TextTidy and valid multiline cleanup is rejected
- Dominant language
- Swift
- Stars
- 4
- Forks
- 17
- Avg merge
- 3h 32m
- Merged PRs (30d)
- 277
Description
## Problem
`TextTidy.collapseSpacing` promises to preserve a model answer's line breaks, but only splits on the LF `Character`. Swift treats CRLF as one character, so CRLF remains inside a chunk and the subsequent whitespace splitting converts it to a space.
On main `5c310667cd35c44abd8bbd26c19b0905471287b4`, equivalent LF and CRLF model responses take different paths: the LF response is accepted, while the CRLF response is altered by cleanup and then rejected for dropping a requested line break.
## Reproduction and evidence
Use the actual `GenerativeTextTransformer` with the repository's scripted `FakeCleanupModel` and a request for:
```text
do we discount new line answer colon we do not discount
```
Have the model return `Do we discount?` followed by the tested newline and `Answer: We do not discount.`. Run the same request through `TransformerRouter` with this transformer followed by `RuleBasedTransformer`.
Observed with the production helpers and engines:
```text
LF:
unwrapped="Do we discount?\nAnswer: We do not discount."
tidied="Do we discount?\nAnswer: We do not discount."
router=foundationModels
result="Do we discount?\nAnswer: We do not discount."
CRLF:
unwrapped="Do we discount?\r\nAnswer: We do not discount."
tidied="Do we discount? Answer: We do not discount."
direct transformer throws outputRejected:
"the rewrite dropped a line break the speaker asked for"
router=rules
result="Do we discount\nanswer: we do not discount."
```
The LF control passes. The CRLF acceptance assertion fails with the rejection above. The 60 existing transformation, rules, spacing, script-guard, Latin-only and warming tests passed in the isolated harness before this probe.
This uses a controlled model response, not a captured Apple Foundation Models response. It proves handling of a valid newline representation at the `CleanupModel` boundary; it does not establish how frequently the current model emits CRLF.
## Location
- [TextTidy.swift](https://github.com/uttrflow/uttrflow-swift/blob/5c310667cd35c44abd8bbd26c19b0905471287b4/Sources/UttrflowAI/TextTidy.swift#L13-L20): LF-only split followed by whitespace collapse.
- [GenerativeTextTransformer.swift](https://github.com/uttrflow/uttrflow-swift/blob/5c310667cd35c44abd8bbd26c19b0905471287b4/Sources/UttrflowAI/GenerativeTextTransformer.swift#L59-L80): unwrapping, spacing cleanup, finishing and guard validation.
- `TextTidyTests.collapseSpacing` tests LF and blank lines, but no CRLF case.
`ResponseUnwrapper` preserves CRLF for this answer; the first observed loss is the spacing helper.
## Impact and acceptance criteria
Priority P2. Correctly laid-out model output is unnecessarily discarded, and the user receives the less polished rules result. The guard prevents silent insertion of the flattened result in this reproduction.
- Treat LF and CRLF consistently when preserving line breaks, including consecutive blank lines and mixed endings.
- Preserve the helper's intended horizontal-whitespace cleanup.
- Add helper and full-transformer regression coverage showing equivalent accepted output for the two model responses.
- Keep the rejection for a model answer that actually omits a requested break.
This is separate from #803, which concerns clipboard formatter-diff line splitting, and #789, which concerns context supplied to suggestions. Neither fixes this dictation-cleanup path.
Contributor guide
Assessment
This issue has not been assessed yet.