Can write but read parquet file with nested arrays

I am trying to read a parquet file in scala using the Avro interface (1.10.). The file was also generated using the same interface.

The data that I am writing looks like this:

 
```java

case class Inner(b: Array[Int])
case class Outer(a: Array[Inner])

val data = Outer(
Array(
Inner(Array(1, 2)),
Inner(Array(3, 4))
)
)
```
 

Using parquet-tools to read read the file looks like this:

 
```java

$ parquet-tools cat /tmp/test.parquet
a:
.array:
..b:
...array = 1
...array = 2
.array:
..b:
...array = 3
...array = 4
```
 

But while trying to read the file I get the following exception:

 

 
```java

 
Exception in thread "main" org.apache.parquet.io.InvalidRecordException: Parquet/Avro schema mismatch: Avro field 'array' not found
at org.apache.parquet.avro.AvroRecordConverter.getAvroField(AvroRecordConverter.java:225)
at org.apache.parquet.avro.AvroRecordConverter.(AvroRecordConverter.java:130)
at org.apache.parquet.avro.AvroRecordConverter.newConverter(AvroRecordConverter.java:279)
at org.apache.parquet.avro.AvroRecordConverter.newConverter(AvroRecordConverter.java:232)
at org.apache.parquet.avro.AvroRecordConverter.access$100(AvroRecordConverter.java:78)
at org.apache.parquet.avro.AvroRecordConverter$AvroCollectionConverter$ElementConverter.(AvroRecordConverter.java:536)
at org.apache.parquet.avro.AvroRecordConverter$AvroCollectionConverter.(AvroRecordConverter.java:486)
at org.apache.parquet.avro.AvroRecordConverter.newConverter(AvroRecordConverter.java:289)
at org.apache.parquet.avro.AvroRecordConverter.(AvroRecordConverter.java:141)
at org.apache.parquet.avro.AvroRecordConverter.(AvroRecordConverter.java:95)
at org.apache.parquet.avro.AvroRecordMaterializer.(AvroRecordMaterializer.java:33)
at org.apache.parquet.avro.AvroReadSupport.prepareForRead(AvroReadSupport.java:138)
at org.apache.parquet.hadoop.InternalParquetRecordReader.initialize(InternalParquetRecordReader.java:183)
at org.apache.parquet.hadoop.ParquetReader.initReader(ParquetReader.java:156)
at org.apache.parquet.hadoop.ParquetReader.read(ParquetReader.java:135)
at raw.runtime.writer.parquet.avro.Lixo$.main(Lixo.scala:78)
at raw.runtime.writer.parquet.avro.Lixo.main(Lixo.scala)

 
```
 

This is the code used to generate this file:
```java

val filename = "/tmp/test.parquet"
val path = Paths.get(filename).toFile
val conf = new Configuration()

val schema: Schema = {

val inner = Schema.createRecord("inner", "some doc", "outer", false,
List(new Schema.Field("b", Schema.createArray(Schema.create(Schema.Type.INT)), "", null: Object)).asJava
)

Schema.createRecord("outer", "", "", false,
List(new Schema.Field("a", Schema.createArray(inner), "", null: Object)).asJava
)
}

val os = new FileOutputStream(path)

val outputFile = new RawParquetOutputFile(os)
val parquetWriter: ParquetWriter[GenericRecord] = AvroParquetWriter.builder[GenericRecord](outputFile)
.withConf(conf)
.withSchema(schema)
.build()

val data = Outer(
Array(
Inner(Array(1, 2)),
Inner(Array(3, 4))
)
)

val record = new GenericData.Record(schema)
val fieldA = schema.getField("a").schema()
val recorData = {
val fieldAType = fieldA.getElementType()
data.a.map { x =>
val innerRecord = new GenericData.Record(fieldAType)
innerRecord.put("b", x.b)
innerRecord
}
}

record.put("a", recorData)
parquetWriter.write(record)
parquetWriter.close()
os.close()
```
 Also if I pass the configuration option 
```java

parquet.avro.add-list-element-records = false

```
I get a different exception:

org.apache.avro.SchemaParseException: Can't redefine: list

 

Am I doing something wrong?

 

 

**Reporter**: [cesar matos](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=Torcato)
#### Related issues:
- [SchemaParseException: Can't redefine: list in AvroIndexedRecordConverter](https://github.com/apache/parquet-java/issues/2239) (is duplicated by)

_{**Note**: *This issue was originally created as [PARQUET-1409](https://issues.apache.org/jira/browse/PARQUET-1409). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*}

Bắt đầu với AvroRecordConverter.java và AvroReadSupport.java tại các vị trí trong stack trace, sau đó xem lại issue liên quan 2239 và tái hiện việc đọc mảng lồng nhau bằng schema và cấu hình được cung cấp. Được xem là hoàn tất khi tệp parquet do ví dụ tạo ra có thể được đọc thông qua giao diện Avro mà không xảy ra lỗi không khớp schema hoặc SchemaParseException.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.