apache / apache/parquet-java

AvroSchemaConverter silently drops typed_value when converting shredded variant schemas

Abierto
#3,477 6 comentarios 0 reacciones 1 asignado Reclamado por @nssalian Ver en GitHub
Type: bug
Lenguaje dominante
Java
Estrellas
3.1k
Forks
1.6k
Merge medio
3 d 12 h
PR fusionados (30 d)
33

Descripción

### Describe the bug, including details regarding any error messages, version, and platform.

I noticed this on the parquet-cli while checking the schema for a shredded parquet file.

Essentially, given this Parquet schema:

```
message table {
required group data (VARIANT(1)) {
required binary metadata;
optional binary value;
optional group typed_value {
required group name {
optional binary value;
optional binary typed_value (STRING);
}
required group age {
optional binary value;
optional int32 typed_value (INTEGER(8,true));
}
}
}
}
```

AvroSchemaConverter.convert() produces:

```
{
"type": "record",
"name": "data",
"fields": [
{"name": "metadata", "type": "bytes"},
{"name": "value", "type": "bytes"}
]
}
```

`typed_value` is missing

Expected: The converter should convert all children of the VARIANT group (including typed_value when present)

Version: 1.18.0-SNAPSHOT

I can add a fix to this myself if there are no objections. If we want to keep this behavior, we should make a note.

### Component(s)

Avro

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.