authorjapps / authorjapps/zerocode

Kafka test with specialized serializer and deserializer

Open
#610 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
1k
Forks
453
Avg merge
7d 2h
Merged PRs (30d)
5

Description

`KafkaConsumerHelper#convertProtobufToJson` and `KafkaProducerHelper#buildProtoMessage` assume that we're using `ByteArrayDeserializer` and `ByteArraySerializer`. They expect the `ConsumerRecord#value()` to be a `byte[]` or return a `byte[]`. This is problematic when using special SerDes that handle the low level parsing/serialization; for example, the SerDes from https://github.com/awslabs/aws-glue-schema-registry or Confluent's SerDes for their Schema Registry. These libraries handle the POJOs directly, i.e. `ConsumerRecord#value()` will be of type `MessageOrBuilder`, and no further parsing (via Reflection) will be needed. There should be a property that allows overriding the default behavior so `KafkaConsumerHelper` and `KafkaProducerHelper` just accept/return POJO of the wire type instead of `byte[]`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.