uttrflow / uttrflow/uttrflow-swift

Meaning guard accepts moving a negation to another clause, reversing the dictated instruction

Open
#815 0 comments 0 reactions 1 assignee Claimed by @naveen-bhatt View on GitHub
Dominant language
Swift
Stars
4
Forks
17
Avg merge
3h 32m
Merged PRs (30d)
277

Description

## Reproduced behavior

The full `GenerativeTextTransformer` accepts this meaning-changing model response on main `5c310667cd35c44abd8bbd26c19b0905471287b4`:

```text
Dictated: we should not approve the design but we should approve the budget
Returned: We should approve the design but we should not approve the budget.
```

The person forbids approving the design and permits approving the budget. The returned text does the reverse. The words and their counts are unchanged; only the position of `not` changed.

## Verification

An isolated test used the repository's actual `GenerativeTextTransformer`, standard cleaning passes and meaning guard, with `FakeCleanupModel` returning the response above. It called `transform` on a real `TransformationRequest` with an English transcription.

```text
accepted=We should approve the design but we should not approve the budget.
Meaning-changing negation relocation was accepted
```

The rejection expectation fails. This is a controlled model response, not an observed hallucination from Apple's model or a test that inserted text into another application. It establishes that the production acceptance boundary admits the reversal.

34 existing context, number and mirrored-guard tests passed. The mirrored guard tests already reject adding/removing a negation; those count-changing cases do not exercise relocation.

## Cause

In [MeaningPreservationGuard.swift](https://github.com/uttrflow/uttrflow-swift/blob/5c310667cd35c44abd8bbd26c19b0905471287b4/Sources/UttrflowAI/MeaningPreservationGuard.swift):

- `grammarVerdict` sends content words to `survivalVerdict` and `placeVerdict`. Here the content words remain in the same order.
- Negation checks compare total counts in the kept and rewritten token sequences. Both contain one negator, so neither the added nor the dropped check rejects.
- `functionWordChurn` compares multisets. Moving the same `not` contributes zero churn.

Although an alignment now exists, it is not used to enforce where this negation belongs.

## Expected / acceptance criteria

Priority P1: the guard is supposed to prevent a model rewrite from reversing the speaker's instruction.

- Reject a negation moved between clauses while allowing harmless case/punctuation and supported contraction changes.
- Add a full-transformer regression using the example above and its inverse, alongside the existing added/dropped-negation cases.
- Ensure a rejected response reaches the deterministic fallback rather than being inserted as model-cleaned text.
- Evaluate the change against the cleanup corpus before merging; a blanket ban on all token movement would need evidence that it preserves legitimate cleanup.

## Related issues and distinction

Closed #188 addressed added negations. Closed #189 addressed moved content words and permissive stem matching. Open #194 concerns substitutions of prepositions/modals within a churn allowance; this reproduction changes no word or multiset count and bypasses the explicit negation checks. Open #229 is the broader guard-design umbrella; this is a concrete remaining acceptance defect in the current aligned implementation and can be tracked beneath it.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.