confluentinc / confluentinc/confluent-kafka-python
OverflowError: Python int too large to convert to C int when using confluent_kafka Avro deserializer
- Dominant language
- Python
- Stars
- 509
- Forks
- 964
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 14
Description
Description
===========
I'm encountering an OverflowError when attempting to deserialize messages using the **confluent_kafka **Avro deserializer in Python. Here's a simplified version of my code:
```
class ConsumerKafka:
def __init__(self, deserializer):
self.deserializer = deserializer
self.consumer = Consumer()
def decode(self, msg_value):
deserialized_data = self.deserializer(msg_value, None)
```
self.serilizer is AvroDeserializer object from onfluent_kafka.schema_registry.avro
when i call the self.deserializer(msg_value, None)
```
Traceback (most recent call last):
File "c:\Users\048115571\Documents\python\TTA\read_from_topic\modules\consumer.py", line 30, in decode
deserialized_data = self.deserializer(msg_value, None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "c:\Users\048115571\AppData\Local\Programs\Python\Python311\Lib\site-packages\confluent_kafka\schema_registry\avro.py", line 429, in __call__
obj_dict = schemaless_reader(payload,
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "fastavro\\_read.pyx", line 1142, in fastavro._read.schemaless_reader
File "fastavro\\_read.pyx", line 1169, in fastavro._read.schemaless_reader
File "fastavro\\_read.pyx", line 748, in fastavro._read._read_data
File "fastavro\\_read.pyx", line 621, in fastavro._read.read_record
File "fastavro\\_read.pyx", line 740, in fastavro._read._read_data
File "fastavro\\_read.pyx", line 558, in fastavro._read.read_union
File "fastavro\\_read.pyx", line 724, in fastavro._read._read_data
File "fastavro\\_read.pyx", line 393, in fastavro._read.read_array
File "fastavro\\_read.pyx", line 748, in fastavro._read._read_data
File "fastavro\\_read.pyx", line 621, in fastavro._read.read_record
File "fastavro\\_read.pyx", line 740, in fastavro._read._read_data
File "fastavro\\_read.pyx", line 558, in fastavro._read.read_union
File "fastavro\\_read.pyx", line 770, in fastavro._read._read_data
File "fastavro\\_logical_readers.pyx", line 22, in fastavro._logical_readers.read_timestamp_millis
File "fastavro\\_logical_readers.pyx", line 24, in fastavro._logical_readers.read_timestamp_millis
OverflowError: Python int too large to convert to C int
```
How to reproduce
================
confluent_avro 1.8.0
confluent-kafka 2.3.0
fastavro 1.9.2
kafka-python 2.0.2
- [ ] Operating system: windows
Checklist
=========
Please provide the following information:
- [ ] confluent-kafka-python and librdkafka version (`confluent_kafka.version()` and `confluent_kafka.libversion()`):
- [ ] Apache Kafka broker version:
- [ ] Client configuration: `{...}`
- [ ] Operating system:
- [ ] Provide client logs (with `'debug': '..'` as necessary)
- [ ] Provide broker log excerpts
- [ ] Critical issue
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.