swiftlang / swiftlang/swift-experimental-string-processing
Regex does not match isolated combining mark as whitespace if preceded by whitespace
Nobody has claimed this yet.
- Dominant language
- Swift
- Stars
- 308
- Forks
- 52
- Avg merge
- 12h 45m
- Merged PRs (30d)
- 2
Description
Description
I believe Regexes should function on Unicode scalars, not on Swift Chars. This is a failure mode: <space>+<combining mark> (such as " ̃") is seen as a single whitespace character, where all other programming languages I know of regard it conceptually as a single whitespace character plus a single non-spacing combining character.
Reproduction
let aTilde = "ã" // \u{0061} + \u{0303}
let aMatch = try! /\S/.firstMatch(in: aTilde)
print(aMatch?.output) // "ã" hm... I would have expected only the scalar 'a' to match
let combiningTilde = "̃" // \u{0303}
let tildeMatch = try! /\S/.firstMatch(in: combiningTilde)
print(tildeMatch?.output) // "̃" correct to me
let spaceWithTilde = " ̃" // space+tilde
let spaceTildeMatch = try! /\S/.firstMatch(in: spaceWithTilde)
print(spaceTildeMatch?.output) // nil, but I would expect \u{0303} to match
Expected behavior
tilde scalar was expected to match regex, since it is not a whitespace codepoint (WS) according to Unicode specification, but non-spacing (Mn)
Environment
5.9
Additional information
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the provided Swift reproduction with the three strings and inspect the regex matching path that handles Swift Chars versus Unicode scalars. Done means \S matches only the non-whitespace combining scalar in the space-plus-tilde case while preserving the other demonstrated behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- swift
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100