Add possibility to use pre-registered Avro schema with the outbox router [DBZ-5706]
- Dominant language
- HTML
- Stars
- 6
- Forks
- 8
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 1
Description
Migrated from [DBZ-5706](https://issues.redhat.com/browse/DBZ-5706)
Hi,
Following this discussion [https://groups.google.com/g/debezium/c/bD-TZ3RsMdY].
The business need is to reflect the process of a developer thinking about an Avro schema for his events to be inserted in an outbox table, registering this schema in the schema registry through CI/CD, and using it to publish with the outbox router ; without the need of having Debezium inferring the schema of the JSONB column content of the outbox table.
As suggested by [~jpechane], I looked for 3rd party SMTs for doing it but :
* I didn't find any
* I would prefer to limit the number of plugins to be deployed on the connect instances
As per the potential implementation, I think having a dedicated column in the outbox table that can be configured in the connectors should be OK. Nevertheless, there are I think several possibilities :
* Either this column represents a schema ID in the schema registry
* Either it represents a schema name + version
I think the first one would be difficult to use because depending on the developer environment, the same functional schema will have different IDs, which should be hard to manage in their application.
Thanks a lot
Contributor guide
Research direction
Start by reviewing the outbox router configuration and the linked schema-registry discussion. Compare the proposed schema ID and schema name plus version approaches, then define what configuration and outbox-table behavior would let the router publish using a pre-registered schema instead of inferring the JSONB content.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kafka
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100