spring-projects / spring-projects/spring-boot

Make it possible and simple to use more than one kafka serializer and deserializer

Open
#43,363 12 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type: enhancement
Dominant language
Java
Stars
81.5k
Forks
42.7k
Avg merge
2d 4h
Merged PRs (30d)
65

Description

We are in the situation where we have both consumers and producers in the code that all use avro, and using config only (in YAML files) worked fine for this.

But now we have a single producer where the topic's consumer expect JSON objects serialized as strings, and we would like to be able to send messages to this topic without throwing out everything we have set up for avro to the other topics we communicate on.

Ideally we would have like to be able to use config to e.g. set a different serializer for a particular record type (key/message type combination) or topic or something. But that's not possible to do today.

So we have to configure producers in code.

But we still would prefer not to configure everything (bootstrap server, schema registry, the SSL config) in code. We would like to use as much as possible of the existing config for this.

Googling found us this stack overflow answer to someone wanting to do the same thing back in 2018 https://stackoverflow.com/a/50310843

But the API used in this example has been deprecated and removed, so we have googled on and landed on https://github.com/spring-projects/spring-boot/issues/39144

So our wish is basically: keep all config but be able to set custom serializers (and possibly deserializers, but we don't need that (yet))

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing Spring Boot's existing Kafka producer configuration and the deprecated API discussed in the linked Stack Overflow answer and spring-projects/spring-boot#39144. Done should preserve shared bootstrap, schema registry, and SSL settings while allowing serializer selection for a particular record type or topic, with coverage for the supported configuration path.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, kafka, spring-boot
Domain
backend, distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.