[VL] LowCopyNettyJniByteInputStream does not work in uniffle netty mode
Open
enhancement
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 85
Description
### Description
Shuffle reader does not use `LowCopyNettyJniByteInputStream` as expected in uniffle netty mode, but still uses `OnHeapJniByteInputStream`.
uniffle client conf:
```
spark.rss.client.type=GRPC_NETTY
```
### Gluten version
None
Contributor guide
Research direction
Start by tracing the shuffle reader's input-stream selection when spark.rss.client.type=GRPC_NETTY, comparing where LowCopyNettyJniByteInputStream and OnHeapJniByteInputStream are chosen. Done means uniffle netty mode selects the low-copy stream as described and the behavior is covered by an appropriate regression check.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- grpc, scala, spark
- Domain
- data-engineering, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100