aio-libs / aio-libs/aiokafka

PY_SSIZE_T_CLEAN macro must be defined for '#' formats

Abierto
#926 1 comentario 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
1.4k
Forks
269
Merge medio
1 d 1 h
PR fusionados (30 d)
6

Descripción

**Describe the bug**
I got this error when trying to produce the message using aiokafka 0.8.1 and Python 3.11.6

This urls might useful for someone who is smarter than me
https://bugs.python.org/issue40943
https://peps.python.org/pep-0353/

Here's the stack trace:
```
[Tue 2023-10-10 11:13:34+0000] ERROR [aiokafka.producer.sender._sender_routine:167] Unexpected error in sender routine
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/sender.py", line 155, in _sender_routine
task.result()
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/sender.py", line 260, in _send_produce_req
await handler.do(node_id)
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/sender.py", line 714, in do
request = self.create_request()
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/sender.py", line 678, in create_request
(tp.partition, batch.get_data_buffer())
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/message_accumulator.py", line 234, in get_data_buffer
return self._builder._build()
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/message_accumulator.py", line 87, in _build
self._buffer = self._builder.build()
^^^^^^^^^^^^^^^^^^^^^
File "aiokafka/record/_crecords/default_records.pyx", line 716, in aiokafka.record._crecords.default_records.DefaultRecordBatchBuilder.build
File "aiokafka/record/_crecords/default_records.pyx", line 701, in aiokafka.record._crecords.default_records.DefaultRecordBatchBuilder._maybe_compress
SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats
[Tue 2023-10-10 11:13:34+0000] ERROR [asyncio.default_exception_handler:1771] Exception in callback Sender._fail_all()
handle: )>
Traceback (most recent call last):
File "/usr/local/lib/python3.11/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/sender.py", line 70, in _fail_all
self._txn_manager.fatal_error(task_exception)
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/transaction_manager.py", line 163, in fatal_error
if self._transaction_waiter.done():
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'done'

```

I've tried to add this lines of code:
```
#ifndef PY_SSIZE_T_CLEAN
#define PY_SSIZE_T_CLEAN
```
to aiokafka/record/_crecords/crc32c.h and aiokafka/record/_crecords/crc32c.c
And it worked, but when I used lz4 compression I got this error:
```
[Tue 2023-10-10 11:13:34+0000] ERROR [aiokafka.producer.sender._sender_routine:167] Unexpected error in sender routine
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/sender.py", line 155, in _sender_routine
task.result()
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/sender.py", line 260, in _send_produce_req
await handler.do(node_id)
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/sender.py", line 714, in do
request = self.create_request()
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/sender.py", line 678, in create_request
(tp.partition, batch.get_data_buffer())
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/message_accumulator.py", line 234, in get_data_buffer
return self._builder._build()
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/message_accumulator.py", line 87, in _build
self._buffer = self._builder.build()
^^^^^^^^^^^^^^^^^^^^^
File "aiokafka/record/_crecords/default_records.pyx", line 716, in aiokafka.record._crecords.default_records.DefaultRecordBatchBuilder.build
File "aiokafka/record/_crecords/default_records.pyx", line 701, in aiokafka.record._crecords.default_records.DefaultRecordBatchBuilder._maybe_compress
SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats
[Tue 2023-10-10 11:13:34+0000] ERROR [asyncio.default_exception_handler:1771] Exception in callback Sender._fail_all()
handle: )>
Traceback (most recent call last):
File "/usr/local/lib/python3.11/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/sender.py", line 70, in _fail_all
self._txn_manager.fatal_error(task_exception)
File "/usr/local/lib/python3.11/site-packages/aiokafka/producer/transaction_manager.py", line 163, in fatal_error
if self._transaction_waiter.done():
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'done'
```

So I guess that helps somewhat but kafka.codecs also has this problem?

**Environment (please complete the following information):**
- aiokafka version: 0.8.1
- kafka-python version: 2.0.2
- Kafka Broker version (`kafka-topics.sh --version`):

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.