authorjapps / authorjapps/zerocode
Kafka test with specialized serializer and deserializer
- Dominant language
- Java
- Stars
- 1k
- Forks
- 453
- Avg merge
- 7d 2h
- Merged PRs (30d)
- 5
Description
`KafkaConsumerHelper#convertProtobufToJson` and `KafkaProducerHelper#buildProtoMessage` assume that we're using `ByteArrayDeserializer` and `ByteArraySerializer`. They expect the `ConsumerRecord#value()` to be a `byte[]` or return a `byte[]`. This is problematic when using special SerDes that handle the low level parsing/serialization; for example, the SerDes from https://github.com/awslabs/aws-glue-schema-registry or Confluent's SerDes for their Schema Registry. These libraries handle the POJOs directly, i.e. `ConsumerRecord#value()` will be of type `MessageOrBuilder`, and no further parsing (via Reflection) will be needed. There should be a property that allows overriding the default behavior so `KafkaConsumerHelper` and `KafkaProducerHelper` just accept/return POJO of the wire type instead of `byte[]`.
Contributor guide
Assessment
This issue has not been assessed yet.