PY_SSIZE_T_CLEAN macro must be defined for '#' formats
- Lenguaje dominante
- Python
- Estrellas
- 1.4k
- Forks
- 269
- Merge medio
- 1 d 1 h
- PR fusionados (30 d)
- 6
Descripción
**Describe the bug**
I got this error when trying to produce the message using aiokafka 0.8.1 and Python 3.11.6
This urls might useful for someone who is smarter than me
https://bugs.python.org/issue40943
https://peps.python.org/pep-0353/
Here's the stack trace:
```
[Tue 2023-10-10 11:13:34+0000] ERROR [aiokafka.producer.sender._sender_routine:167] Unexpected error in sender routine
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/sender.py", line 155, in _sender_routine
task.result()
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/sender.py", line 260, in _send_produce_req
await handler.do(node_id)
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/sender.py", line 714, in do
request = self.create_request()
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/sender.py", line 678, in create_request
(tp.partition, batch.get_data_buffer())
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/message_accumulator.py", line 234, in get_data_buffer
return self._builder._build()
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/message_accumulator.py", line 87, in _build
self._buffer = self._builder.build()
^^^^^^^^^^^^^^^^^^^^^
File "aiokafka/record/_crecords/default_records.pyx", line 716, in aiokafka.record._crecords.default_records.DefaultRecordBatchBuilder.build
File "aiokafka/record/_crecords/default_records.pyx", line 701, in aiokafka.record._crecords.default_records.DefaultRecordBatchBuilder._maybe_compress
SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats
[Tue 2023-10-10 11:13:34+0000] ERROR [asyncio.default_exception_handler:1771] Exception in callback Sender._fail_all()
handle: )>
Traceback (most recent call last):
File "/usr/local/lib/python3.11/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/sender.py", line 70, in _fail_all
self._txn_manager.fatal_error(task_exception)
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/transaction_manager.py", line 163, in fatal_error
if self._transaction_waiter.done():
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'done'
```
I've tried to add this lines of code:
```
#ifndef PY_SSIZE_T_CLEAN
#define PY_SSIZE_T_CLEAN
```
to aiokafka/record/_crecords/crc32c.h and aiokafka/record/_crecords/crc32c.c
And it worked, but when I used lz4 compression I got this error:
```
[Tue 2023-10-10 11:13:34+0000] ERROR [aiokafka.producer.sender._sender_routine:167] Unexpected error in sender routine
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/sender.py", line 155, in _sender_routine
task.result()
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/sender.py", line 260, in _send_produce_req
await handler.do(node_id)
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/sender.py", line 714, in do
request = self.create_request()
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/sender.py", line 678, in create_request
(tp.partition, batch.get_data_buffer())
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/message_accumulator.py", line 234, in get_data_buffer
return self._builder._build()
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/message_accumulator.py", line 87, in _build
self._buffer = self._builder.build()
^^^^^^^^^^^^^^^^^^^^^
File "aiokafka/record/_crecords/default_records.pyx", line 716, in aiokafka.record._crecords.default_records.DefaultRecordBatchBuilder.build
File "aiokafka/record/_crecords/default_records.pyx", line 701, in aiokafka.record._crecords.default_records.DefaultRecordBatchBuilder._maybe_compress
SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats
[Tue 2023-10-10 11:13:34+0000] ERROR [asyncio.default_exception_handler:1771] Exception in callback Sender._fail_all()
handle: )>
Traceback (most recent call last):
File "/usr/local/lib/python3.11/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/sender.py", line 70, in _fail_all
self._txn_manager.fatal_error(task_exception)
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/transaction_manager.py", line 163, in fatal_error
if self._transaction_waiter.done():
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'done'
```
So I guess that helps somewhat but kafka.codecs also has this problem?
**Environment (please complete the following information):**
- aiokafka version: 0.8.1
- kafka-python version: 2.0.2
- Kafka Broker version (`kafka-topics.sh --version`):
Guía de contribución
Evaluación
Este issue todavía no se ha evaluado.