Cannot import an extension module if its path is not encodable in UTF-8

Bug report

Importing an extension module fails with UnicodeEncodeError if its path contains characters unencodable in UTF-8, e.g. surrogate escapes of a file name undecodable in the filesystem encoding.

$ .../nd\udcffwt/python -c "import _json"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
UnicodeEncodeError: 'utf-8' codec can't encode character '\udcff' in position 45: surrogates not allowed

As a result, CPython cannot be run in a directory whose name contains undecodable bytes, and cannot even be built there -- make fails in checksharedmods. On 3.13 and 3.14 every import of a shared extension module from such a path fails.

The cause is the key of the extensions cache: hashtable_key_from_2_strings() in Python/import.c encodes the module file name with _PyUnicode_AsUTF8NoNUL(). It was added in gh-105699.

The raw content of the strings (UCS1, UCS2 or UCS4) can be used for the key instead of the UTF-8 encoding.

Linked PRs

• gh-155249

從 Python/import.c 中的 hashtable_key_from_2_strings() 開始，issue 將擴充模組檔名的 UTF-8 編碼確定為原因。檢查連結的 PR gh-155249，以及報告中描述的 checksharedmods 失敗。完成結果應包括：能夠從包含無法解碼的檔名字節的路徑成功匯入和建置擴充模組。