apache / apache/pulsar-client-python
[Bug] Segmentation fault when properties exceed ~100kb
- 主要語言
- Python
- 星號
- 75
- 分支
- 53
- PR 合併指標
- 30 天內沒有已合併 PR
描述
Hi everyone, i'm running an event driven app and i use apache pulsar as backbone. I use message's properties to exchange metadata between the services.
I noted that, when the properties exceed ~100kb the client gives `Segmentation fault (core dumped)` (sometimes `corrupted double-linked list` instead)
> I haven't done a proper test on the size, so take the size as a guideline
I was able to simulate the behaviour in a notebook:
```py
import pulsar
import sys
import json
# need to serialize to json nested object because client accepts properties only
# of type (self: _pulsar.MessageBuilder, arg0: str, arg1: str)
nested_dict = json.dumps({
"foo": "bar"
})
big_dict = {
f"key{i}": nested_dict for i in range(3000)
}
print(sys.getsizeof(big_dict)) # 103856
client = pulsar.Client('pulsar://localhost:6650')
producer = client.create_producer('test-topic')
producer.send(content="test-message".encode(), properties=big_dict) # segfault
```
Unfortunately i don't have details in the stacktrace:
```
In [4]: producer.send(content="test-message".encode(), properties=big_dict)
Segmentation fault (core dumped)
```
Pulsar client version is 3.5.0.
貢獻指南
研究方向
首先,使用回報的大型 properties payload,透過 pulsar.Client、create_producer 和 producer.send 執行 notebook 重現,然後擷取缺少的 stack trace。當相同的 payload 不再導致 segmentation fault 或 corrupted double-linked list,且大型 properties 案例有 coverage 時,即表示完成。
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- python
- 領域
- distributed-systems
- Issue 類型
- 缺陷
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 活躍度
- 停滯
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100