`chompIf` / `chompWhile` hand the predicate half a surrogate pair

Package: gren-lang/core
Module: String.Parser.Advanced (and so String.Parser, which delegates to it)
Function: isSubChar
Version observed: gren-lang/core 7.4.2

Summary

When the next character is outside the Basic Multilingual Plane, the predicate
given to chompIf, chompWhile or variable is called with the leading
surrogate code unit rather than with the character. isSubChar decodes the
surrogate pair into a local named uniChar and then tests the undecoded unit
anyway; uniChar is bound and never used.

The effect is that any parser whose predicate asks what a character is stops
dead at the first emoji, mathematical alphanumeric, or CJK extension character.
It does not fail loudly -- chompWhile simply chomps nothing there, and the
parse fails later at a place that has nothing to do with the cause.

Reproduction

import Char
import String.Parser.Advanced as P

chomp : (Char -> Bool) -> String -> Result String String
chomp pred src =
    when P.run (P.getChompedString (P.chompWhile pred)) {} src is
        Ok text -> Ok text
        Err _ -> Err "failed"

The last row is the whole bug in one line: the predicate matches 0xD83D, which
is the high surrogate of U+1F600 and is not a character at all, and matching it
chomps the entire two-unit sequence.

For contrast, everything outside the parser gets this right:

String.toArray "😀" |> Array.map Char.toCode
--> [ 128512 ]     -- 0x1F600, the code point, correctly

So Char is a code point everywhere in core except here.

Cause

In String.Parser.Advanced:

isSubChar : (Char -> Bool) -> Int -> String -> Int
isSubChar pred offset str =
  when String.getUnit offset str is
    Nothing ->
      -1

    Just unit ->
      if Bitwise.and (Char.toCode unit) 0xF800 == 0xD800 then
        when String.popFirst (String.sliceUnits offset (offset + 2) str) is
          Just { first = uniChar } ->
            if pred unit then          -- <-- should be `pred uniChar`
              offset + 2

            else
              -1

          Nothing ->
            -1

      else
        ...

The surrogate branch does the right thing to get the character: it slices the
two code units and pops the first Char off, which combines them. It then
ignores the result. A compiler warning for the unused uniChar would have
caught it.

The offset arithmetic is correct -- offset + 2 -- which is why
chompWhile (\_ -> True) works and why this is invisible until a predicate
actually inspects the character.

I suggest also adding a unit test along the lines of:

P.run (P.getChompedString (P.chompWhile (\c -> Char.toCode c > 0xFFFF))) {} "😀"
--> Ok "😀"

Impact

Any parser that classifies characters by hand -- which is most of them, since
that is what chompWhile is for -- silently mis-parses text containing
non-BMP characters. Found while writing a TOML parser: the official TOML test
suite has valid/comment/nonascii.toml, whose comment ends with U+10000 and
U+10FFFF, and valid/multibyte.toml, which uses mathematical alphanumerics
(𝓼𝓽𝓻𝓲𝓷𝓰).

Start at String.Parser.Advanced.isSubChar, which is also used through String.Parser. Check the surrogate branch and add a regression test for chompWhile with a non-BMP character such as 😀. Done means character-inspecting predicates receive the decoded code point while the existing chomp behavior remains correct.