Breaking change in Avro 1.12.1 affects parquet-avro users: logical types now deserialize to different Java types
- Vorherrschende Sprache
- Java
- Sterne
- 3.1k
- Forks
- 1.6k
- Ø Merge
- 3 T. 12 Std.
- Gemergte PRs (30 T.)
- 33
Beschreibung
## Summary
Avro 1.12.1 introduces a breaking change that affects `parquet-avro` users. Multiple logical types are now deserialized to their proper Java types instead of their underlying primitive types when using `ReflectData`, causing `ClassCastException` in existing code.
## Background
- **Upstream issue:** https://issues.apache.org/jira/browse/AVRO-3989
- **Pull request:** https://github.com/apache/avro/pull/3354
- **Avro version affected:** 1.12.1 (released as a patch version)
## Problem
The fix in AVRO-3989 changed the deserialization behavior for ALL logical types. When using within parquet-avro `ReflectData` or `SpecificData` classes, these fields now return their logical Java type objects instead of their underlying primitive types from `GenericRecord`.
### Affected logical types and conversions:
| Logical Type | Old Type (≤1.12.0) | New Type (1.12.1) |
|--------------|-------------------|-------------------|
| `timestamp-millis` | `Long` | `Instant` |
| `timestamp-micros` | `Long` | `Instant` |
| `date` | `Integer` | `LocalDate` |
| `time-millis` | `Integer` | `LocalTime` |
| `time-micros` | `Long` | `LocalTime` |
| `local-timestamp-millis` | `Long` | `LocalDateTime` |
| `local-timestamp-micros` | `Long` | `LocalDateTime` |
| `uuid` | `String` | `UUID` |
| `decimal` | `ByteBuffer` | `BigDecimal` |
This breaks existing code that expects primitive values:
```java
// This used to work in Avro 1.12.0
Long timestamp = (Long) genericRecord.get("timestamp_field");
Integer date = (Integer) genericRecord.get("date_field");
String uuid = (String) genericRecord.get("uuid_field");
// In Avro 1.12.1, all of these throw ClassCastException
// because values are now Instant, LocalDate, UUID, etc.
```
## Impact
- **Widespread breakage:** Affects anyone using parquet-avro with ANY logical types
- Code that has worked for years suddenly breaks after a patch version update
- The breaking nature wasn't clearly documented in Avro's changelog
- This is a significant behavior change that affects the entire logical type system
## Heads up for future avro updates
I apologize if this isn't the right place to report this, please feel free to redirect me if there's a better venue for this discussion. My main goal is to ensure that when parquet-java does consider updating Avro, the team and users are aware of this significant behavioral change.
## For Current Users
If you're using parquet-avro and explicitly upgrading Avro to 1.12.1+ in your project, be aware of this breaking change. You'll need to update code that reads logical type fields from `GenericRecord`.
Beitragsleitfaden
Für dieses Repository ist kein Beitragsleitfaden indexiert
Rechercherichtung
Beginnen Sie mit dem AVRO-3989-Link des Issues und dem hier beschriebenen Verhalten von ReflectData, SpecificData und GenericRecord; es wird keine parquet-java-Datei oder kein Test genannt. Vergleichen Sie die Verarbeitung der aufgeführten logischen Typen in Avro 1.12.0 und 1.12.1 und klären Sie anschließend mit den Maintainer:innen, welche Änderung oder Dokumentation erwartet wird, bevor Sie festlegen, wann die Aufgabe als erledigt gilt.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- java
- Bereich
- data-engineering
- Issue-Typ
- Bug
- Schwierigkeit
- 5/5
- Geschätzter Aufwand
- Über eine Woche
- Aktivitätsstatus
- Ruhig
- Klarheit
- Muss geklärt werden
- Anfängerfreundlichkeit
- 25/100