apache / apache/gluten

VeloxRssSortShuffleReaderDeserializer: infinite loop on EOS mid-page and corrupted deserialization on multi-window Presto pages

Open
#12,985 1 comment 0 reactions 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Backend

VL (Velox)

### Bug description

### Description

We hit two bugs in the rss_sort shuffle reader (`VeloxRssSortShuffleReaderDeserializer::VeloxInputStream`) in our production environment with Celeborn (rss_sort + hash partitioning). Both reproduce on current main — regression tests submitted in #12983 fail.

### Bug 1: infinite loop on EOS mid-page

`GlutenByteInputStream::readBytes()` drives `next(true)` in a `for(;;)` loop. When a page header declares more bytes than the stream actually holds (truncated partition data), the reader hits EOS mid-page — but `VeloxInputStream::next()` ignores its `throwIfPastEnd` argument and silently returns on EOS, so the loop never exits.

Symptom in production: the task appears stuck (thread in RUNNABLE, no progress, no error surfaced).

CI evidence from #12983:

```
C++ exception with description "possible infinite loop: Read() returned 0 for 100 consecutive calls" thrown in the test body.
```

(the test's fake stream cuts the loop short after 100 consecutive EOS reads instead of hanging)

Additionally, `next()` stores the `Read()` result into an unsigned `offset_` without a signed guard, so a negative return from a buggy upstream corrupts `setRange` instead of failing fast.

### Bug 2: corrupted deserialization when a page spans multiple read windows

`VeloxInputStream` is a single-window refill stream: each `next()` overwrites the sole `ByteRange` with serde assumes stable multi-range data — for nested types it does `tellp()` → pre-scan the page →`seekp()` back, and it scans the payload via `nextView()` for checksum verification. When an uncompressed page (nested structs) spans multiple ~1MB read windows, the serde's backward seek reads window data already overwritten by a
refill, corrupting the stream.

CI evidence from #12983:

```
Exception: VeloxRuntimeError
Reason: (-673841146 vs. 0) Invalid serialized string size: -673841146
Function: readLengthPrefixedString
# 3 ...readLengthPrefixedString(ByteInputStream*)
# 6 ...PrestoVectorSerde::deserialize(...)
# 9 gluten::VeloxRssSortShuffleReaderDeserializer::next()
```

### Impact

- Bug 1: task hangs forever with no error — requires external timeout to kill, hard to diagnose from logs.
- Bug 2: wrong data / spurious deserialization errors for pages larger than one read window (~1MB) containing nested types.

### Fix

We have a fix ready and will submit a PR shortly (referencing this issue).

### Gluten version

main branch, Gluten-1.3

### Spark version

Spark-3.5.x

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

Contributor guide

Open the contributing guide

Research direction

Start with VeloxRssSortShuffleReaderDeserializer::VeloxInputStream and the regression tests referenced in #12983. Trace next(), readBytes(), and nextView() through EOS handling and multi-window refills. Done means truncated pages fail promptly and pages spanning read windows deserialize without corruption.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.