Avro "fixed" field causes ClassCastException
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 142
Description
**Describe the bug**
If you have an Avro "fixed" field in a record (or presumably if you try to produce a "fixed" field directly), you get a ClassCastException when producing a message.
```
java.lang.ClassCastException: class org.apache.pulsar.examples.Fixed cannot be cast to class org.apache.pulsar.shade.org.apache.avro.generic.GenericFixed (org.apache.pulsar.examples.Fixed and org.apache.pulsar.shade.org.apache.avro.generic.GenericFixed are in unnamed module of loader 'app')
```
(full traceback below)
I slightly tweaked @sijie 's [`pulsar-avro-schema-example`](https://github.com/sijie/pulsar-avro-schema-example) project to demonstrate the issue: [patricklucas/pulsar-avro-schema-example](https://github.com/patricklucas/pulsar-avro-schema-example).
(I updated dependencies, and then added a [fixed field](https://github.com/patricklucas/pulsar-avro-schema-example/blob/master/src/main/avro/fixed.avsc) to the [TwitterSchema](https://github.com/patricklucas/pulsar-avro-schema-example/blob/master/src/main/avro/twitter.avsc) record. Additionally, I added code to serialize the message to JSON before producing, to demonstrate that Avro itself handles the message correctly)
It's possible this is unsupported in some way by Pulsar, but I couldn't find anything in the documentation about such limitations, so I thought I'd ask here.
**To Reproduce**
Steps to reproduce the behavior:
1. Clone the project [patricklucas/pulsar-avro-schema-example](https://github.com/patricklucas/pulsar-avro-schema-example)
2. Run [TweetProducer](https://github.com/patricklucas/pulsar-avro-schema-example/blob/master/src/main/java/org/apache/pulsar/examples/TweetProducer.java), passing a Pulsar service URL as a single argument
3. Observe error
**Expected behavior**
The message should be serialized successfully.
**Additional context**
Full traceback:
```
org.apache.pulsar.client.api.SchemaSerializationException: java.lang.ClassCastException: class org.apache.pulsar.examples.Fixed cannot be cast to class org.apache.pulsar.shade.org.apache.avro.generic.GenericFixed (org.apache.pulsar.examples.Fixed and org.apache.pulsar.shade.org.apache.avro.generic.GenericFixed are in unnamed module of loader 'app')
at org.apache.pulsar.client.impl.schema.writer.AvroWriter.write(AvroWriter.java:55)
at org.apache.pulsar.client.impl.schema.AbstractStructSchema.encode(AbstractStructSchema.java:50)
at org.apache.pulsar.client.impl.TypedMessageBuilderImpl.value(TypedMessageBuilderImpl.java:178)
at org.apache.pulsar.client.impl.ProducerBase.send(ProducerBase.java:63)
at org.apache.pulsar.examples.TweetProducer.main(TweetProducer.java:43)
Caused by: java.lang.ClassCastException: class org.apache.pulsar.examples.Fixed cannot be cast to class org.apache.pulsar.shade.org.apache.avro.generic.GenericFixed (org.apache.pulsar.examples.Fixed and org.apache.pulsar.shade.org.apache.avro.generic.GenericFixed are in unnamed module of loader 'app')
at org.apache.pulsar.shade.org.apache.avro.generic.GenericDatumWriter.writeFixed(GenericDatumWriter.java:339)
at org.apache.pulsar.shade.org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:147)
at org.apache.pulsar.shade.org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:82)
at org.apache.pulsar.shade.org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:158)
at org.apache.pulsar.shade.org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:206)
at org.apache.pulsar.shade.org.apache.avro.specific.SpecificDatumWriter.writeField(SpecificDatumWriter.java:101)
at org.apache.pulsar.shade.org.apache.avro.reflect.ReflectDatumWriter.writeField(ReflectDatumWriter.java:186)
at org.apache.pulsar.shade.org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:195)
at org.apache.pulsar.shade.org.apache.avro.specific.SpecificDatumWriter.writeRecord(SpecificDatumWriter.java:83)
at org.apache.pulsar.shade.org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:130)
at org.apache.pulsar.shade.org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:82)
at org.apache.pulsar.shade.org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:158)
at org.apache.pulsar.shade.org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
at org.apache.pulsar.client.impl.schema.writer.AvroWriter.write(AvroWriter.java:53)
... 4 more
```
Contributor guide
Research direction
Start with AvroWriter.java at the failing write call in the traceback, then reproduce the problem with TweetProducer from the linked pulsar-avro-schema-example project. Compare the fixed.avsc and twitter.avsc schemas and the serialization path; done means a record containing an Avro fixed field is serialized and produced successfully without the ClassCastException.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100