apache / apache/texera

Hash partitioning crashes on LARGE_BINARY keys

Open
#8,185 1 comment 0 reactions 1 assignee Claimed by @carloea2 View on GitHub
Dominant language
Scala
Stars
314
Forks
187
Avg merge
1d 21h
Merged PRs (30d)
214

Description

### Feature Summary

Python tuple hashing has no mapping for `LARGE_BINARY`. A hash shuffle using a large-binary field reaches `Tuple.__hash__` and raises `KeyError` before selecting a downstream worker.

Expected behavior:

Python hashes `LARGE_BINARY` values from their URI using the same UTF-16 and `Objects.hash` semantics as Scala.

Reproduction evidence:

Create a hash partition with two receivers and use a `LARGE_BINARY` key containing `s3://bucket/object`.

Before: partitioning raises `KeyError: AttributeType.LARGE_BINARY`
After: the tuple hashes to the JDK-verified value -46745592 and selects worker 0

ASCII URI, supplementary Unicode URI, and null values are covered.

Version and commit evidence:

Current `main` at 70c21145887920528d7d5540e3fb790b43e8b759.

**Commit Hash (Optional)**

`70c21145887920528d7d5540e3fb790b43e8b759`

### Proposed Solution or Design

Expected behavior:

### Affected Area

Workflow Engine (Amber)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.