apache / apache/texera

Scala range shuffle misroutes negative decimal keys

Open
#8,171 1 comment 0 reactions 1 assignee Claimed by @carloea2 View on GitHub
Dominant language
Scala
Stars
314
Forks
187
Avg merge
1d 21h
Merged PRs (30d)
214

Description

### Feature Summary

The Scala range shuffle converts `DOUBLE` keys to `Long` before calculating their bucket. Fractional values are truncated toward zero, so negative decimals near a bucket boundary can be sent to a different worker than the Python range shuffle.

Expected behavior:

Both execution engines preserve decimal values while choosing the range bucket.

Reproduction evidence:

Use two receivers over the inclusive range -10 to 9 and partition a `DOUBLE` key of -0.5.

Before: Scala bucket 1, Python bucket 0
After: Scala bucket 0, Python bucket 0

A key of 0.0 remains in bucket 1, confirming the actual boundary is unchanged.

Version and commit evidence:

Current `main` at 70c21145887920528d7d5540e3fb790b43e8b759.

**Commit Hash (Optional)**

`70c21145887920528d7d5540e3fb790b43e8b759`

### Proposed Solution or Design

Expected behavior:

### Affected Area

Workflow Engine (Amber)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.