Problem with geomesa_pyspark in version 3.2.0

Problem with geomesa_pyspark in version 3.2.0

I successfully ingested example csv data from examples folder.

But when i try to run pyspark sql query it wont show results (error bellow code, looks like some versions issue).
I use version setup explained in envirionment and for geomesa_pyspark i use this code.
import geomesa_pyspark conf = geomesa_pyspark.configure( jars=['/spark/jars/geomesa-accumulo-spark-runtime-accumulo2_2.12-3.2.0.jar'], packages=['geomesa_pyspark','pytz'], spark_home='/spark').\ setAppName('HelloGeomesaApp').setMaster('spark://spark-master:7077') conf.set('spark.driver.host','172.21.0.12') from pyspark.sql import SparkSession spark = SparkSession.builder.config(conf=conf).getOrCreate() spark.sparkContext.pythonExec = "/usr/local/bin/python3" params = { "accumulo.instance.id": "accumulo", "accumulo.zookeepers": "zookeeper", "accumulo.user": "root", "accumulo.password": "GisPwd", "accumulo.catalog": "geomesa.example" } feature = "example-csv" df = ( spark .read .format("geomesa") .options(**params) .option("geomesa.feature", feature) .load() ) df.createOrReplaceTempView("tbl") spark.sql("show tables").show() spark.sql(""" select count(*) from tbl """).show() Py4JJavaError: An error occurred while calling o75.showString. : java.lang.ArrayIndexOutOfBoundsException: 28499 at com.thoughtworks.paranamer.BytecodeReadingParanamer$ClassReader.accept(BytecodeReadingParanamer.java:563) at com.thoughtworks.paranamer.BytecodeReadingParanamer$ClassReader.access$200(BytecodeReadingParanamer.java:338) at com.thoughtworks.paranamer.BytecodeReadingParanamer.lookupParameterNames(BytecodeReadingParanamer.java:103) at com.thoughtworks.paranamer.CachingParanamer.lookupParameterNames(CachingParanamer.java:79) at com.fasterxml.jackson.module.scala.introspect.BeanIntrospector$.getCtorParams(BeanIntrospector.scala:45) at com.fasterxml.jackson.module.scala.introspect.BeanIntrospector$.$anonfun$apply$1(BeanIntrospector.scala:59) at com.fasterxml.jackson.module.scala.introspect.BeanIntrospector$.$anonfun$apply$1$adapted(BeanIntrospector.scala:59) at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:245) at scala.collection.Iterator.foreach(Iterator.scala:941) at scala.collection.Iterator.foreach$(Iterator.scala:941) at scala.collection.AbstractIterator.foreach(Iterator.scala:1429) at scala.collection.IterableLike.foreach(IterableLike.scala:74) at scala.collection.IterableLike.foreach$(IterableLike.scala:73) at scala.collection.AbstractIterable.foreach(Iterable.scala:56) at scala.collection.TraversableLike.flatMap(TraversableLike.scala:245) at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:242) at scala.collection.AbstractTraversable.flatMap(Traversable.scala:108) at com.fasterxml.jackson.module.scala.introspect.BeanIntrospector$.findConstructorParam$1(BeanIntrospector.scala:59) at com.fasterxml.jackson.module.scala.introspect.BeanIntrospector$.$anonfun$apply$19(BeanIntrospector.scala:181) at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238) at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36) at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33) at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198) at scala.collection.TraversableLike.map(TraversableLike.scala:238) at scala.collection.TraversableLike.map$(TraversableLike.scala:231) at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:198) at com.fasterxml.jackson.module.scala.introspect.BeanIntrospector$.$anonfun$apply$14(BeanIntrospector.scala:175) at com.fasterxml.jackson.module.scala.introspect.BeanIntrospector$.$anonfun$apply$14$adapted(BeanIntrospector.scala:174) at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:245) at scala.collection.immutable.List.foreach(List.scala:392) at scala.collection.TraversableLike.flatMap(TraversableLike.scala:245) at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:242) at scala.collection.immutable.List.flatMap(List.scala:355) at com.fasterxml.jackson.module.scala.introspect.BeanIntrospector$.apply(BeanIntrospector.scala:174)

Original JIRA Issue: https://geomesa.atlassian.net/browse/GEOMESA-3103

Key: GEOMESA-3103
Type: Bug
Priority: Major
Status: To Do
Resolution: Incomplete
Reporter: Dinko
Created: Fri, 16 Jul 2021 06:57:22 -0400
Updated: Wed, 21 Jul 2021 03:06:14 -0400
Resolved: Wed, 21 Jul 2021 02:15:06 -0400
Affects Version(s): 3.2.0
Parent: GEOMESA-1569
JIRA Labels: geomesa_pyspark

Start by reproducing the failure from the geomesa_pyspark.configure setup, the specified GeoMesa runtime JAR, SparkSession creation, and the SQL count query. Inspect the reported Py4JJavaError and Paranamer/Jackson stack trace alongside the environment and version configuration. Done means the example-csv query runs and returns results without the ArrayIndexOutOfBoundsException.