confluentinc / confluentinc/confluent-kafka-python

ProtobufDeserializer raises ValueError("MessageDescriptor not found in file") with a message type generated by the protocol buffer compiler

Open
#1,390 0 comments 0 reactions 1 assignee Claimed by @pranavrth View on GitHub
component:schema-registry investigate further
Dominant language
Python
Stars
509
Forks
964
Avg merge
2d 2h
Merged PRs (30d)
14

Description

Description
===========
Using ProtobufDeserializer with a message type generated by the protocol buffer compiler. Results in a `ValueError("MessageDescriptor not found in file")`. Was able to reproduce with the protobuf_consumer.py example.

```
Traceback (most recent call last):
File "/app/confluent-kafka-python/examples/protobuf_consumer.py", line 90, in
main(parser.parse_args())
File "/app/confluent-kafka-python/examples/protobuf_consumer.py", line 44, in main
protobuf_deserializer = ProtobufDeserializer(user_pb2.User,
File "/usr/local/lib/python3.10/site-packages/confluent_kafka/schema_registry/protobuf.py", line 496, in __init__
self._msg_index = _create_msg_index(descriptor)
File "/usr/local/lib/python3.10/site-packages/confluent_kafka/schema_registry/protobuf.py", line 112, in _create_msg_index
raise ValueError("MessageDescriptor not found in file")
ValueError: MessageDescriptor not found in file
```

How to reproduce
================
Dockerfile to reproduce from protobuf_consumer example.
```
FROM python:3.10.5-buster as python-base

RUN apt-get update -y
RUN mkdir /app
WORKDIR /app

RUN git clone https://github.com/confluentinc/confluent-kafka-python.git
WORKDIR /app/confluent-kafka-python/examples
RUN pip install -r requirements.txt

CMD ["python", "protobuf_consumer.py", "-b", "localhost", "-s", "any", "-t", "help", "-g", "example"]
```

Checklist
=========
Please provide the following information:

- [x] confluent-kafka-python and librdkafka version (`confluent_kafka.version()` and `confluent_kafka.libversion()`):
confluent_kafka.version(): ('1.9.0', 17367040)
confluent_kafka.libversion(): ('1.9.0', 17367295)
- [ ] Apache Kafka broker version:
- [x] Client configuration: `provided example`
- [x] Operating system: Debian
- [x] Provide client logs (with `'debug': '..'` as necessary)
- [x] Provide broker log excerpts
- [x] Critical issue

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.