swiftlang / swiftlang/swift-experimental-string-processing

Regex does not match isolated combining mark as whitespace if preceded by whitespace

Open
#724 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Swift
Stars
308
Forks
52
Avg merge
12h 45m
Merged PRs (30d)
2

Description

Description

I believe Regexes should function on Unicode scalars, not on Swift Chars. This is a failure mode: <space>+<combining mark> (such as " ̃") is seen as a single whitespace character, where all other programming languages I know of regard it conceptually as a single whitespace character plus a single non-spacing combining character.

Reproduction
let aTilde = "ã" // \u{0061} + \u{0303}
let aMatch = try! /\S/.firstMatch(in: aTilde) 
print(aMatch?.output) // "ã" hm... I would have expected only the scalar 'a' to match
let combiningTilde = "̃" // \u{0303}
let tildeMatch = try! /\S/.firstMatch(in: combiningTilde)
print(tildeMatch?.output) // "̃" correct to me
let spaceWithTilde = " ̃" // space+tilde
let spaceTildeMatch = try! /\S/.firstMatch(in: spaceWithTilde)
print(spaceTildeMatch?.output) // nil, but I would expect \u{0303} to match
Expected behavior

tilde scalar was expected to match regex, since it is not a whitespace codepoint (WS) according to Unicode specification, but non-spacing (Mn)

Environment

5.9

Additional information

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided Swift reproduction with the three strings and inspect the regex matching path that handles Swift Chars versus Unicode scalars. Done means \S matches only the non-whitespace combining scalar in the space-plus-tilde case while preserving the other demonstrated behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.