AbsaOSS / AbsaOSS/ABRiS

Version 6.4.0 failing for Spark 3.5.0

未關閉
#357 2 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Scala
星號
242
分支
84
平均合併
10 小時 12 分鐘
30 天內合併 PR
2

描述

Abris Version: 6.4.0
Spark Version: 3.5.0
Databricks Runtime: 14.3

Failing with error:
Cause: java.lang.NoClassDefFoundError: org/apache/spark/sql/internal/SQLConf$LegacyBehaviorPolicy$
at org.apache.spark.sql.avro.AvroDeserializer.(AvroDeserializer.scala:62)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.spark.sql.avro.AbrisAvroDeserializer$$anonfun$1.applyOrElse(AbrisAvroDeserializer.scala:39)
at org.apache.spark.sql.avro.AbrisAvroDeserializer$$anonfun$1.applyOrElse(AbrisAvroDeserializer.scala:37)
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:38)
at scala.util.Failure.recover(Try.scala:234)
at org.apache.spark.sql.avro.AbrisAvroDeserializer.(AbrisAvroDeserializer.scala:37)

In Spark 3.4.0 "LegacyBehaviorPolicy" is available in same file as here:
https://github.com/apache/spark/blob/branch-3.4/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala

However in Spark 3.5.0 "LegacyBehaviorPolicy" is available and used in package:
https://github.com/apache/spark/blob/branch-3.5/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
But is available new package:
https://github.com/apache/spark/blob/branch-3.5/sql/api/src/main/scala/org/apache/spark/sql/internal/LegacyBehaviorPolicy.scala

Could this be issue for the above error?

貢獻指南

這個儲存庫沒有索引到貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。