[BUG] PostgresDebeziumAvroPayload throws ClassCastException on BYTES columns after lazy Avro deserialization
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
### Describe the problem you faced
Merging a Debezium Postgres change event that carries a non-null `BYTES` column (or a union containing `BYTES`) fails with a `ClassCastException`:
```
java.lang.ClassCastException: org.apache.hudi.common.model.SerializableIndexedRecord
cannot be cast to org.apache.avro.generic.GenericData$Record
at org.apache.hudi.common.model.debezium.PostgresDebeziumAvroPayload.containsBytesToastedValues(PostgresDebeziumAvroPayload.java:143)
```
`PostgresDebeziumAvroPayload` casts records to the concrete class `GenericData.Record` in `containsBytesToastedValues` and in `mergeToastedValuesIfPresent`. Since #13987 ("perf: Lazy deserialization of Avro indexed record"), `BaseAvroPayload.getRecord` returns a `SerializableIndexedRecord` whenever the payload's record does not already carry the exact `Schema` instance being merged with, which is the ordinary case in production and after any Kryo round trip. `SerializableIndexedRecord` implements `GenericRecord` but does not extend `GenericData.Record`, so the cast fails.
That PR converted the casts in `containsStringToastedValues` to the `GenericRecord` interface but left the two in `containsBytesToastedValues` and one in `mergeToastedValuesIfPresent`, which is why only tables with a `BYTES` column are affected.
The existing `TestPostgresDebeziumAvroPayload#testMergeWithToastedValues` misses this because it constructs the record and merges with the very same `Schema` instance, so the payload hands back the original `GenericData.Record` and never takes the lazy path.
### To Reproduce
1. Ingest a Postgres table through the Debezium source with `hoodie.datasource.write.payload.class=org.apache.hudi.common.model.debezium.PostgresDebeziumAvroPayload`.
2. Give the table a `BYTES` (or nullable `BYTES`) column with a non-null value.
3. Apply an update whose LSN is ahead of the record already in storage, so the incoming event wins the LSN comparison and the TOAST check runs.
### Expected behavior
The TOAST check works against any `GenericRecord` implementation, and the merge preserves the previous value for toasted columns.
### Environment Description
* Hudi version: master (`c63c9bfa79f` and later)
### Additional context
Affects any `IndexedRecord` implementation that is not a `GenericData.Record`, so the same latent problem exists on `branch-0.x` even though the lazy-deserialization path that triggers it is only on master.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.