apache / apache/texera

Python binary hashes disagree with Scala for non-ASCII bytes

Open
#8,179 1 comment 0 reactions 1 assignee Claimed by @carloea2 View on GitHub
Dominant language
Scala
Stars
314
Forks
187
Avg merge
1d 21h
Merged PRs (30d)
214

Description

### Feature Summary

Python iterates `bytes` as unsigned values from 0 to 255, while Java array hashing promotes signed bytes from -128 to 127. The Java-compatible Python tuple hash passes binary bytes through unchanged, so non-ASCII binary keys can choose different hash-shuffle workers in Python and Scala.

Expected behavior:

Python binary tuple hashes use Java signed-byte semantics.

Reproduction evidence:

Use three receivers and a binary key containing the single byte `0xFF`.

Before: Python hash 317 selects worker 2, Scala hash 61 selects worker 1
After: both hashes are 61 and both select worker 1

The signed boundary is covered with `0x7F`, `0x80`, and `0xFF`.

Version and commit evidence:

Current `main` at 70c21145887920528d7d5540e3fb790b43e8b759.

**Commit Hash (Optional)**

`70c21145887920528d7d5540e3fb790b43e8b759`

### Proposed Solution or Design

Expected behavior:

### Affected Area

Workflow Engine (Amber)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.