Previous exception in task: org.apache.thrift.transport.TSocket.<init>(Lorg/apache/thrift/TConfiguration;Ljava/lang/String;II)V
- Vorherrschende Sprache
- Java
- Sterne
- 6.4k
- Forks
- 1.2k
- Ø Merge
- 1 T. 23 Std.
- Gemergte PRs (30 T.)
- 115
Beschreibung
spark2.4 访问 IoTDB-1.31.1 发生异常,thrift-0.14.2,详细错误入下,
org.apache.spark.util.TaskCompletionListenerException: null
Previous exception in task: org.apache.thrift.transport.TSocket.(Lorg/apache/thrift/TConfiguration;Ljava/lang/String;II)V
org.apache.iotdb.rpc.RpcTransportFactory.getTransport(RpcTransportFactory.java:82)
org.apache.iotdb.jdbc.IoTDBConnection.openTransport(IoTDBConnection.java:481)
org.apache.iotdb.jdbc.IoTDBConnection.(IoTDBConnection.java:113)
org.apache.iotdb.jdbc.IoTDBDriver.connect(IoTDBDriver.java:64)
java.sql.DriverManager.getConnection(DriverManager.java:664)
java.sql.DriverManager.getConnection(DriverManager.java:247)
org.apache.iotdb.spark.db.IoTDBRDD$$anon$1.(IoTDBRDD.scala:74)
org.apache.iotdb.spark.db.IoTDBRDD.compute(IoTDBRDD.scala:54)
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
org.apache.spark.scheduler.Task.run(Task.scala:121)
org.apache.spark.executor.Executor$TaskRunner$$anonfun$11.apply(Executor.scala:407)
org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1408)
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:413)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
at org.apache.spark.TaskContextImpl.invokeListeners(TaskContextImpl.scala:138)
at org.apache.spark.TaskContextImpl.markTaskCompleted(TaskContextImpl.scala:116)
at org.apache.spark.scheduler.Task.run(Task.scala:131)
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$11.apply(Executor.scala:407)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1408)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:413)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Beitragsleitfaden
Rechercherichtung
Beginnen Sie mit org.apache.iotdb.rpc.RpcTransportFactory.getTransport in RpcTransportFactory.java:82 und verfolgen Sie den Verbindungspfad durch IoTDBConnection.java:481 und IoTDBRDD.scala:74. Vergleichen Sie die erwarteten Abhängigkeiten von Spark 2.4, IoTDB 1.31.1 und thrift-0.14.2 und reproduzieren Sie anschließend den gemeldeten Job-Fehler; abgeschlossen ist die Arbeit, wenn der Spark-Zugriff nicht mehr den TSocket-Konstruktorfehler auslöst.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- java, spark
- Bereich
- databases, distributed-systems
- Issue-Typ
- Bug
- Schwierigkeit
- 4/5
- Geschätzter Aufwand
- 3-5 Tage
- Aktivitätsstatus
- Veraltet
- Klarheit
- Muss geklärt werden
- Anfängerfreundlichkeit
- 30/100