apache / apache/texera

Fractional doubles can route to different range partitions

Open
#8,255 1 comment 0 reactions 1 assignee Claimed by @carloea2 View on GitHub
Dominant language
Scala
Stars
314
Forks
187
Avg merge
1d 21h
Merged PRs (30d)
214

Description

### Feature Summary

The JVM range partitioner truncates double keys to a long before calculating their bucket, while the Python partitioner uses the raw fractional value. With ranges that cross zero, finite negative fractions can therefore route to different workers in the two engines.

Before: key -0.5 in range -10 through 9 routes to A in Python and B in Scala

After: Python truncates the finite key to zero and routes to B

Expected behavior:

Python should truncate finite double keys toward zero before range routing, matching Scala, while retaining the existing endpoint behavior for infinities.

Reproduction evidence:

On `50321e403c82df299a13deb50a7f9849dd93bdba`, range-shuffle key `-0.5` across receivers A and B with minimum -10 and maximum 9.

Observed result:

```text
python_index=0 receiver=A scala_truncated_index=1
```

Version and commit evidence:

1.3.0-incubating-SNAPSHOT (main)

**Commit Hash (Optional)**

`50321e403c82df299a13deb50a7f9849dd93bdba`

### Proposed Solution or Design

After: Python truncates the finite key to zero and routes to B

Expected behavior:

### Affected Area

Workflow Engine (Amber)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.