confluentinc / confluentinc/confluent-kafka-python

AvroConsumer — access to key & value schemas

Open
#289 5 comments 4 reactions 0 assignees View on GitHub
component:schema-registry enhancement priority:high status:help-wanted
Dominant language
Python
Stars
509
Forks
964
Avg merge
2d 2h
Merged PRs (30d)
14

Description

Description
===========

AvroConsumer happily returns decoded Avro messages, but drops all reference to the original Avro schema/identifier. The schema is super-useful in some cases (eg. Debezium, where it contains the source database table schema).

One (relatively straightforward) solution is to:
1. update `avro.MessageSerializer.decode_message()` to return a `(schema_id, payload)` tuple
2. have `AvroConsumer.poll()` wrap the `Message` in a Python `AvroMessage` subclass which has additional `.key_schema_id` and `.value_schema_id` attributes. Could also do this in C.
3. expose the schema registry client via a `AvroConsumer.get_schema(schema_id)` method

Checklist
=========
Please provide the following information:

- [x] confluent-kafka-python and librdkafka version (`confluent_kafka.version()` and `confluent_kafka.libversion()`): 0.11.0
- [ ] Apache Kafka broker version:
- [ ] Client configuration: `{...}`
- [ ] Operating system:
- [ ] Provide client logs (with `'debug': '..'` as necessary)
- [ ] Provide broker log excerpts
- [ ] Critical issue

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.