`compare` on `String` orders UTF-16 code units, not characters

Repository: gren-lang/core
Found against: gren 0.6.6, gren-lang/core 7.4.2, node 22

This is a minor issue, but it's a surprise in how compare works.

Summary

Every other String function in core is character-oriented: count, slice,
toArray, foldl and the rest walk codepoints, and Char.toCode reports a
codepoint. compare is the exception — it hands the two strings to JavaScript
<, which compares UTF-16 code units.

Astral Unicode characters are code points above U+FFFF,
such as emoji (e.g., 😀 U+1F600) or symbols like 𝌆 (U+1D306),
and in UTF‑16 they are encoded as surrogate pairs.

When comparing UTF-16, because a lead surrogate is
0xD800–0xDBFF, every astral character sorts below every character in
U+E000–U+FFFF:

compare "\u{FFFE}" "\u{10000}" == GT   -- both characters are one codepoint
compare "\u{10000}" "\u{FFFE}" == LT

The result is a consistent total order, just not the order the rest of the API
implies, so nothing raises and nothing looks wrong. Array.sort, Dict and
Set consult compare and nothing else, so they inherit it: a Dict String v
iterates its keys in UTF-16 order.

Strings of ASCII, Latin-1 or ordinary BMP text are unaffected, which is why this
does not show up in practice — the two orders diverge only when a string
containing an astral character (emoji, most historic scripts, mathematical
alphanumerics, and the higher CJK extensions) is compared against one starting
in U+E000–U+FFFF.

Reproduction

module Main exposing (probe)

import Dict


probe : Array String
probe =
    let
        codes s =
            String.toArray s
                |> Array.map (\c -> String.fromInt (Char.toCode c))
                |> String.join "+"

        strings =
            [ "\u{10000}", "\u{FFFE}", "\u{10FFFF}", "\u{0041}" ]
    in
    [ Array.map codes (Array.sort strings) |> String.join ","
    , Array.foldl (\k d -> Dict.set k 0 d) Dict.empty strings
        |> Dict.keys
        |> Array.map codes
        |> String.join ","
    ]

U+FFFF is deliberately absent from the reproduction: that one literal is
miscompiled for an unrelated reason
(compiler#384), and one bug
per report.

Cause

core/src/Gren/Kernel/Utils.js:

function _Utils_cmp(x, y) {
  if (typeof x !== "object") {
    return x === y ? /*EQ*/ 0 : x < y ? /*LT*/ -1 : /*GT*/ 1;
  }
  …

A String is a JavaScript string, so it takes the first branch, and < on
JavaScript strings is defined as a lexicographic comparison of UTF-16 code
units. _Utils_compare, _Utils_lt and friends all route through it, and so
does every Dict and Set operation.

Whether compare should be codepoint-ordered at all is a documentation
question as much as an implementation one: nothing in Basics.compare's docs
says which it is. But the rest of the String API answers it — count and
slice are in characters, yet Dict String v iteration order depends
on UTF-16.

A minimal fix is to simply document this behavior.

Start with core/src/Gren/Kernel/Utils.js and the documentation for Basics.compare, then run the supplied reproduction using the astral and U+E000–U+FFFF strings. Done means the ordering behavior and its effect on Array.sort, Dict, and Set are resolved and covered by the appropriate documentation or tests.