Py_UNICODE_TOUPPER() and friends do not return the simple case mappings

Bug report

Py_UNICODE_TOUPPER(), Py_UNICODE_TOLOWER() and Py_UNICODE_TOTITLE() return the first character of the full case mapping when it is longer than one character.

UnicodeData.txt leaves the simple uppercase of 'ß' (LATIN SMALL LETTER SHARP S) empty, which means the character is its own simple uppercase; the full uppercase "SS" is in SpecialCasing.txt. Py_UNICODE_TOUPPER() returns 'S'. The same for 'ǰ' -> 'J', 'fi' -> 'F' and 'ΐ' -> 'Ι'.

For 27 Greek letters with ypogegrammeni the UCD does define a simple uppercase, and it is not the first character of the full one:

1F80;GREEK SMALL LETTER ALPHA WITH PSILI AND YPOGEGRAMMENI;Ll;0;L;1F00 0345;;;;N;;;1F88;;1F88
1F80; 1F80; 1F88; 1F08 0399; # GREEK SMALL LETTER ALPHA WITH PSILI AND YPOGEGRAMMENI

The simple uppercase of 'ᾀ' is 'ᾈ' (U+1F88), the full one is Ἀ + Ι, and the macro returns Ἀ (U+1F08). Tools/unicode/makeunicodedata.py stores only the full mappings for a character which has a SpecialCasing.txt entry, so the simple ones are not in the table at all.

102 characters are affected for the uppercase, 48 for the titlecase, and one (LATIN CAPITAL LETTER I WITH DOT ABOVE, whose simple lowercase is 'i') for the lowercase. All of them are Unicode 1.1, and no character added since has joined them.

This is a side effect of b2bf01d824e (bpo-12736, 3.3), which added the full mappings: before it the fields held the simple mappings.

_sre is the only user in CPython, and it compensates: _sre.unicode_iscased() reports whether the simple lowercase or uppercase differs from the character, which is right for 'ß' only because the returned 'S' differs from it. Fixing the mappings requires comparing the full mappings there instead.

Linked PRs

• gh-156519

从 Tools/unicode/makeunicodedata.py 开始，该文件目前为 SpecialCasing.txt 中的条目存储完整映射，然后检查 _sre unicode_iscased() 的补偿逻辑。完成的标准是：保留受影响字符的简单大小写映射，并且 _sre 仍使用完整映射正确确定大小写。