apache / apache/pinot

Buffer overflow exception in `VarByteChunkSVForwardIndexWriter` of Spark data push flow

Open
#8,638 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

Our Spark push flow is failing with a table using var length dictionary, but the hadoop job succeeded on the same table. The problem
```
03-05-2022 21:11:29 PDT validate-pinot-code-spark-avro INFO - Caused by: java.nio.BufferOverflowException
03-05-2022 21:11:29 PDT validate-pinot-code-spark-avro INFO - at java.nio.DirectByteBuffer.put(DirectByteBuffer.java:363)
03-05-2022 21:11:29 PDT validate-pinot-code-spark-avro INFO - at java.nio.ByteBuffer.put(ByteBuffer.java:859)
03-05-2022 21:11:29 PDT validate-pinot-code-spark-avro INFO - at org.apache.pinot.segment.local.io.writer.impl.VarByteChunkSVForwardIndexWriter.putBytes(VarByteChunkSVForwardIndexWriter.java:101)
03-05-2022 21:11:29 PDT validate-pinot-code-spark-avro INFO - at org.apache.pinot.segment.local.io.writer.impl.VarByteChunkSVForwardIndexWriter.putString(VarByteChunkSVForwardIndexWriter.java:92)
03-05-2022 21:11:29 PDT validate-pinot-code-spark-avro INFO - at org.apache.pinot.segment.local.segment.creator.impl.fwd.SingleValueVarByteRawIndexCreator.putString(SingleValueVarByteRawIndexCreator.java:109)
03-05-2022 21:11:29 PDT validate-pinot-code-spark-avro INFO - at org.apache.pinot.segment.local.segment.creator.impl.SegmentColumnarIndexCreator.indexRow(SegmentColumnarIndexCreator.java:377)
03-05-2022 21:11:29 PDT validate-pinot-code-spark-avro INFO - at org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl.build(SegmentIndexCreationDriverImpl.java:244)
03-05-2022 21:11:29 PDT validate-pinot-code-spark-avro INFO - at
03-05-2022 21:11:29 PDT validate-pinot-code-spark-avro INFO - at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:219)
03-05-2022 21:11:29 PDT validate-pinot-code-spark-avro INFO - at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:219)
03-05-2022 21:11:29 PDT validate-pinot-code-spark-avro INFO - at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:929)
03-05-2022 21:11:29 PDT validate-pinot-code-spark-avro INFO - at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:929)
03-05-2022 21:11:29 PDT validate-pinot-code-spark-avro INFO - at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2094)
03-05-2022 21:11:29 PDT validate-pinot-code-spark-avro INFO - at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2094)
03-05-2022 21:11:29 PDT validate-pinot-code-spark-avro INFO - at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
03-05-2022 21:11:29 PDT validate-pinot-code-spark-avro INFO - at org.apache.spark.scheduler.Task.run(Task.scala:109)
03-05-2022 21:11:29 PDT validate-pinot-code-spark-avro INFO - at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:429)
```
The problem can be traced back to at least Commit 21632dadb8cd2d8b77aec523a758d73

Contributor guide

Open the contributing guide

Research direction

Start at VarByteChunkSVForwardIndexWriter.putBytes (line 101) and follow calls through putString, SingleValueVarByteRawIndexCreator.putString, and SegmentColumnarIndexCreator.indexRow. Reproduce the Spark push flow with a var-length dictionary table and inspect how the buffer size is determined. Done means the affected push completes without BufferOverflowException, with behavior validated for this path.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spark
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.