confluentinc / confluentinc/confluent-kafka-python
AvroConsumer — access to key & value schemas
- Dominant language
- Python
- Stars
- 509
- Forks
- 964
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 14
Description
Description
===========
AvroConsumer happily returns decoded Avro messages, but drops all reference to the original Avro schema/identifier. The schema is super-useful in some cases (eg. Debezium, where it contains the source database table schema).
One (relatively straightforward) solution is to:
1. update `avro.MessageSerializer.decode_message()` to return a `(schema_id, payload)` tuple
2. have `AvroConsumer.poll()` wrap the `Message` in a Python `AvroMessage` subclass which has additional `.key_schema_id` and `.value_schema_id` attributes. Could also do this in C.
3. expose the schema registry client via a `AvroConsumer.get_schema(schema_id)` method
Checklist
=========
Please provide the following information:
- [x] confluent-kafka-python and librdkafka version (`confluent_kafka.version()` and `confluent_kafka.libversion()`): 0.11.0
- [ ] Apache Kafka broker version:
- [ ] Client configuration: `{...}`
- [ ] Operating system:
- [ ] Provide client logs (with `'debug': '..'` as necessary)
- [ ] Provide broker log excerpts
- [ ] Critical issue
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.