Can write but read parquet file with nested arrays

I am trying to read a parquet file in scala using the Avro interface (1.10.). The file was also generated using the same interface.

The data that I am writing looks like this:

 
```java

case class Inner(b: Array[Int])
case class Outer(a: Array[Inner])

val data = Outer(
Array(
Inner(Array(1, 2)),
Inner(Array(3, 4))
)
)
```
 

Using parquet-tools to read read the file looks like this:

 
```java

$ parquet-tools cat /tmp/test.parquet
a:
.array:
..b:
...array = 1
...array = 2
.array:
..b:
...array = 3
...array = 4
```
 

But while trying to read the file I get the following exception:

 

 
```java

 
Exception in thread "main" org.apache.parquet.io.InvalidRecordException: Parquet/Avro schema mismatch: Avro field 'array' not found
at org.apache.parquet.avro.AvroRecordConverter.getAvroField(AvroRecordConverter.java:225)
at org.apache.parquet.avro.AvroRecordConverter.(AvroRecordConverter.java:130)
at org.apache.parquet.avro.AvroRecordConverter.newConverter(AvroRecordConverter.java:279)
at org.apache.parquet.avro.AvroRecordConverter.newConverter(AvroRecordConverter.java:232)
at org.apache.parquet.avro.AvroRecordConverter.access$100(AvroRecordConverter.java:78)
at org.apache.parquet.avro.AvroRecordConverter$AvroCollectionConverter$ElementConverter.(AvroRecordConverter.java:536)
at org.apache.parquet.avro.AvroRecordConverter$AvroCollectionConverter.(AvroRecordConverter.java:486)
at org.apache.parquet.avro.AvroRecordConverter.newConverter(AvroRecordConverter.java:289)
at org.apache.parquet.avro.AvroRecordConverter.(AvroRecordConverter.java:141)
at org.apache.parquet.avro.AvroRecordConverter.(AvroRecordConverter.java:95)
at org.apache.parquet.avro.AvroRecordMaterializer.(AvroRecordMaterializer.java:33)
at org.apache.parquet.avro.AvroReadSupport.prepareForRead(AvroReadSupport.java:138)
at org.apache.parquet.hadoop.InternalParquetRecordReader.initialize(InternalParquetRecordReader.java:183)
at org.apache.parquet.hadoop.ParquetReader.initReader(ParquetReader.java:156)
at org.apache.parquet.hadoop.ParquetReader.read(ParquetReader.java:135)
at raw.runtime.writer.parquet.avro.Lixo$.main(Lixo.scala:78)
at raw.runtime.writer.parquet.avro.Lixo.main(Lixo.scala)

 
```
 

This is the code used to generate this file:
```java

val filename = "/tmp/test.parquet"
val path = Paths.get(filename).toFile
val conf = new Configuration()

val schema: Schema = {

val inner = Schema.createRecord("inner", "some doc", "outer", false,
List(new Schema.Field("b", Schema.createArray(Schema.create(Schema.Type.INT)), "", null: Object)).asJava
)

Schema.createRecord("outer", "", "", false,
List(new Schema.Field("a", Schema.createArray(inner), "", null: Object)).asJava
)
}

val os = new FileOutputStream(path)

val outputFile = new RawParquetOutputFile(os)
val parquetWriter: ParquetWriter[GenericRecord] = AvroParquetWriter.builder[GenericRecord](outputFile)
.withConf(conf)
.withSchema(schema)
.build()

val data = Outer(
Array(
Inner(Array(1, 2)),
Inner(Array(3, 4))
)
)

val record = new GenericData.Record(schema)
val fieldA = schema.getField("a").schema()
val recorData = {
val fieldAType = fieldA.getElementType()
data.a.map { x =>
val innerRecord = new GenericData.Record(fieldAType)
innerRecord.put("b", x.b)
innerRecord
}
}

record.put("a", recorData)
parquetWriter.write(record)
parquetWriter.close()
os.close()
```
 Also if I pass the configuration option 
```java

parquet.avro.add-list-element-records = false

```
I get a different exception:

org.apache.avro.SchemaParseException: Can't redefine: list

 

Am I doing something wrong?

 

 

**Reporter**: [cesar matos](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=Torcato)
#### Related issues:
- [SchemaParseException: Can't redefine: list in AvroIndexedRecordConverter](https://github.com/apache/parquet-java/issues/2239) (is duplicated by)

_{**Note**: *This issue was originally created as [PARQUET-1409](https://issues.apache.org/jira/browse/PARQUET-1409). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*}

Commencez par AvroRecordConverter.java et AvroReadSupport.java aux emplacements indiqués dans la stack trace, puis examinez l’issue associée 2239 et reproduisez la lecture du tableau imbriqué à l’aide du schéma et de la configuration fournis. C’est terminé lorsque le fichier parquet généré par l’exemple peut être lu via l’interface Avro sans incompatibilité de schéma ni SchemaParseException.

Rédigé par le modèle d'indexation à partir du texte de l'issue.