apache / apache/parquet-java

Breaking change in Avro 1.12.1 affects parquet-avro users: logical types now deserialize to different Java types

Aperta
#3,515 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Java
Stelle
3.1k
Fork
1.6k
Merge medio
3g 12h
PR unite (30g)
33

Descrizione

## Summary

Avro 1.12.1 introduces a breaking change that affects `parquet-avro` users. Multiple logical types are now deserialized to their proper Java types instead of their underlying primitive types when using `ReflectData`, causing `ClassCastException` in existing code.

## Background

- **Upstream issue:** https://issues.apache.org/jira/browse/AVRO-3989
- **Pull request:** https://github.com/apache/avro/pull/3354
- **Avro version affected:** 1.12.1 (released as a patch version)

## Problem

The fix in AVRO-3989 changed the deserialization behavior for ALL logical types. When using within parquet-avro `ReflectData` or `SpecificData` classes, these fields now return their logical Java type objects instead of their underlying primitive types from `GenericRecord`.

### Affected logical types and conversions:

| Logical Type | Old Type (≤1.12.0) | New Type (1.12.1) |
|--------------|-------------------|-------------------|
| `timestamp-millis` | `Long` | `Instant` |
| `timestamp-micros` | `Long` | `Instant` |
| `date` | `Integer` | `LocalDate` |
| `time-millis` | `Integer` | `LocalTime` |
| `time-micros` | `Long` | `LocalTime` |
| `local-timestamp-millis` | `Long` | `LocalDateTime` |
| `local-timestamp-micros` | `Long` | `LocalDateTime` |
| `uuid` | `String` | `UUID` |
| `decimal` | `ByteBuffer` | `BigDecimal` |

This breaks existing code that expects primitive values:

```java
// This used to work in Avro 1.12.0
Long timestamp = (Long) genericRecord.get("timestamp_field");
Integer date = (Integer) genericRecord.get("date_field");
String uuid = (String) genericRecord.get("uuid_field");

// In Avro 1.12.1, all of these throw ClassCastException
// because values are now Instant, LocalDate, UUID, etc.
```

## Impact

- **Widespread breakage:** Affects anyone using parquet-avro with ANY logical types
- Code that has worked for years suddenly breaks after a patch version update
- The breaking nature wasn't clearly documented in Avro's changelog
- This is a significant behavior change that affects the entire logical type system

## Heads up for future avro updates

I apologize if this isn't the right place to report this, please feel free to redirect me if there's a better venue for this discussion. My main goal is to ensure that when parquet-java does consider updating Avro, the team and users are aware of this significant behavioral change.

## For Current Users

If you're using parquet-avro and explicitly upgrading Avro to 1.12.1+ in your project, be aware of this breaking change. You'll need to update code that reads logical type fields from `GenericRecord`.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Direzione di ricerca

Inizia dal link AVRO-3989 dell’issue e dal comportamento di ReflectData, SpecificData e GenericRecord descritto qui; non viene indicato alcun file o test di parquet-java. Confronta la gestione dei tipi logici elencati in Avro 1.12.0 e 1.12.1, quindi stabilisci con i maintainer quale modifica o documentazione è prevista prima di definire quando il lavoro può considerarsi completato.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
java
Ambito
data-engineering
Tipo di issue
Bug
Difficoltà
5/5
Tempo stimato
Più di una settimana
Stato di attività
Tranquilla
Chiarezza
Da chiarire
Idoneità per principianti
25/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.