SplitUtils.getRangeLength can return a negative number
- Dominant language
- Java
- Stars
- 1.2k
- Forks
- 487
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 13
Description
**Describe the bug**
The SplitUtils.getRangeLength() method can return a negative number (due to overflow) for certain Ranges. This became a problem for us using Spark 3.2.x where the default behavior is to ignore empty splits - they actually drop InputSplits where getLength() returns <= 0.
**To Reproduce**
Here's a sample range:
```
Key begin = new Key("aaaaaaa");
Key end = new Key(new byte[] {
(byte) 0xFF,
(byte) 0xFF,
(byte) 0xFF,
(byte) 0xFF,
(byte) 0xFF,
(byte) 0xFF,
(byte) 0xFF
});
Range range = new Range(begin, end);
System.out.println(SplitUtils.getRangeLength(range));
```
That will print out an overflowed Long that has been flipped to a massively negative number.
**Expected behavior**
This should return a number > 0.
Contributor guide
Research direction
Locate SplitUtils.getRangeLength() and reproduce the supplied Range example to inspect how the length is calculated. Add regression coverage for this boundary case and verify that the result is greater than zero, so Spark does not treat the split as empty.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100