confluentinc / confluentinc/confluent-kafka-python

Enhancement - use schema registry to validate provided schemas for Protobuf and Json deserialisers

Open
#1,210 0 comments 0 reactions 0 assignees View on GitHub
component:schema-registry enhancement
Dominant language
Python
Stars
509
Forks
964
Avg merge
2d 2h
Merged PRs (30d)
14

Description

Description
===========

Not an issue but a possible enhancement request.

Currently the protobuf and json deserialisers do not interact with the schema registry (because they don't need to). However it might be useful to optionally check the schema registry to confirm that the provided schema is compatible with the schema used for serialisation?

Looking at the code it seems reasonably easy to implement by adapting the code from the Avro deserialiser. If @edenhill or others think it's useful I'm happy to do a pr.

How to reproduce
================

Checklist
=========
Please provide the following information:

- [ ] confluent-kafka-python and librdkafka version (`confluent_kafka.version()` and `confluent_kafka.libversion()`):
- [ ] Apache Kafka broker version:
- [ ] Client configuration: `{...}`
- [ ] Operating system:
- [ ] Provide client logs (with `'debug': '..'` as necessary)
- [ ] Provide broker log excerpts
- [ ] Critical issue

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the existing Avro deserialiser implementation, then compare the protobuf and JSON deserialiser entry points mentioned in the issue. Define how optional schema-registry compatibility checks should behave for both formats; done means the checks work without changing the default deserialiser behavior and are covered by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.