Kafka Avro Classloading Issue in Quarkustest - SerializationException
- Dominant language
- Java
- Stars
- 302
- Forks
- 232
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 108
Description
### Bug description
In testing trying to consume data for 2 different Avro schemata leads to an exception.
-----------------------
We are using
QuarkusCamel: 3.27.0
but got this bug also in 3.20.3
Currently we are using confluenct registry, using apicurio did not change the behavior.
-----------------------
Trying to consume data for 2 different Avro schemata leads to an exception:
`org.apache.kafka.common.errors.SerializationException: Could not find class RestCallTrace specified in writer's schema whilst finding reader's schema for a SpecificRecord.`
The Route is built up like:
```
from("direct:kafka")
.routeId("FromDirect")
.process(Routes::transformToRestCallTraceOne)
.to(getKafkaFirst())
.process(Routes::transformToRestCallTraceTwo)
.to(getKafkaSecond());
```
using 2 Kafka Topics and a total of 2 Avro Schemata involved. One Schema is used in a topic exclusively.
Tests are failing consuming data from the second kafka topic.
---
Seems that the
`org.apache.avro.specific.SpecificData#classCache`
in
`org.apache.avro.specific.SpecificData#getClass`
is filled using the wrong classloader while writing the data.
In following consuming cached entry is faulty and the consuming fails.
---
A reproducer is attached.
[avro-reproducer.zip](https://github.com/user-attachments/files/23548356/avro-reproducer.zip)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the attached avro-reproducer.zip and inspect org.apache.avro.specific.SpecificData#classCache and getClass, focusing on how the classloader is populated while writing data for the two Kafka topics. Reproduce the failure with the two Avro schemata; done means consuming data from the second topic no longer raises SerializationException.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, kafka
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100