apache / apache/parquet-java

AvroSchemaConverter silently drops typed_value when converting shredded variant schemas

Offen
#3,477 6 Kommentare 0 Reaktionen 1 zugewiesene Person Beansprucht von @nssalian Auf GitHub ansehen
Type: bug
Vorherrschende Sprache
Java
Sterne
3.1k
Forks
1.6k
Ø Merge
3 T. 12 Std.
Gemergte PRs (30 T.)
33

Beschreibung

### Describe the bug, including details regarding any error messages, version, and platform.

I noticed this on the parquet-cli while checking the schema for a shredded parquet file.

Essentially, given this Parquet schema:

```
message table {
required group data (VARIANT(1)) {
required binary metadata;
optional binary value;
optional group typed_value {
required group name {
optional binary value;
optional binary typed_value (STRING);
}
required group age {
optional binary value;
optional int32 typed_value (INTEGER(8,true));
}
}
}
}
```

AvroSchemaConverter.convert() produces:

```
{
"type": "record",
"name": "data",
"fields": [
{"name": "metadata", "type": "bytes"},
{"name": "value", "type": "bytes"}
]
}
```

`typed_value` is missing

Expected: The converter should convert all children of the VARIANT group (including typed_value when present)

Version: 1.18.0-SNAPSHOT

I can add a fix to this myself if there are no objections. If we want to keep this behavior, we should make a note.

### Component(s)

Avro

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.