java.lang.UnsupportedOperationException from SparkExecutor

Nobody has claimed this yet.

Got an exception from SparkExecutor:

Exception in thread “main” org.apache.spark.SparkException: Job aborted due to stage failure: Task 140 in stage 2.0 failed 1 times, most recent failure: Lost task 140.0 in stage 2.0 (TID 402, localhost): java.lang.UnsupportedOperationException
at java.util.AbstractList.add(AbstractList.java:148)
at java.util.AbstractList.add(AbstractList.java:108)
at java.util.AbstractCollection.addAll(AbstractCollection.java:344)
at org.apache.tinkerpop.gremlin.spark.process.computer.payload.ViewIncomingPayload.mergeViewIncomingPayload(ViewIncomingPayload.java:85)
at org.apache.tinkerpop.gremlin.spark.process.computer.payload.ViewIncomingPayload.mergePayload(ViewIncomingPayload.java:100)
at org.apache.tinkerpop.gremlin.spark.process.computer.SparkExecutor.lambda$executeVertexProgramIteration$fb06c361$1(SparkExecutor.java:148)
at org.apache.spark.api.java.JavaPairRDD$$anonfun$toScalaFunction2$1.apply(JavaPairRDD.scala:1012)
at org.apache.spark.util.collection.ExternalAppendOnlyMap$$anonfun$3.apply(ExternalAppendOnlyMap.scala:149)
at org.apache.spark.util.collection.ExternalAppendOnlyMap$$anonfun$3.apply(ExternalAppendOnlyMap.scala:148)
at org.apache.spark.util.collection.AppendOnlyMap.changeValue(AppendOnlyMap.scala:144)
at org.apache.spark.util.collection.SizeTrackingAppendOnlyMap.changeValue(SizeTrackingAppendOnlyMap.scala:32)
at org.apache.spark.util.collection.ExternalAppendOnlyMap.insertAll(ExternalAppendOnlyMap.scala:161)
at org.apache.spark.Aggregator.combineCombinersByKey(Aggregator.scala:58)
at org.apache.spark.shuffle.BlockStoreShuffleReader.read(BlockStoreShuffleReader.scala:83)
at org.apache.spark.rdd.ShuffledRDD.compute(ShuffledRDD.scala:98)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Seems the logic in line 95 would leads to the failure: https://github.com/apache/tinkerpop/blob/master/spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/process/computer/payload/ViewIncomingPayload.java

If the method mergePayload get called multiple times, and the first one is empty, it will replace with Collections.emptyList() result, thus lead to the failure since the empty list is immutable constant. The reducer function in SparkExecutor is supposed to be called multiple times

Start with spark-gremlin/src/main/java/org/apache/tinkerpop/gremlin/spark/process/computer/payload/ViewIncomingPayload.java, especially mergePayload and mergeViewIncomingPayload, then inspect the reducer call in SparkExecutor.java. Reproduce the reported SparkExecutor failure involving repeated reducer calls and confirm that the job completes without UnsupportedOperationException when an initial payload is empty.

Written by the indexing model from the issue text.