Python Unicode hashes disagree with Scala for supplementary characters
- Dominant language
- Scala
- Stars
- 314
- Forks
- 187
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 214
Description
### Feature Summary
Java string hashing iterates UTF-16 code units, while Python tuple hashing iterates Unicode code points. These representations differ for supplementary characters, so equal string keys can choose different hash-shuffle workers in Python and Scala.
Expected behavior:
Python string tuple hashes use Java UTF-16 code-unit semantics.
Reproduction evidence:
Use two receivers and the string key represented by `U+1F600`.
Before: Python hash 128543 selects worker 1, Scala hash 1772930 selects worker 0
After: both hashes are 1772930 and both select worker 0
ASCII, BMP Unicode, and supplementary Unicode are covered.
Version and commit evidence:
Current `main` at 70c21145887920528d7d5540e3fb790b43e8b759.
**Commit Hash (Optional)**
`70c21145887920528d7d5540e3fb790b43e8b759`
### Proposed Solution or Design
Expected behavior:
### Affected Area
Workflow Engine (Amber)
Contributor guide
Assessment
This issue has not been assessed yet.