apache / apache/pulsar-client-python
Pulsar Identity SerDe behaviour and documentation
- 主要語言
- Python
- 星號
- 75
- 分支
- 53
- PR 合併指標
- 30 天內沒有已合併 PR
描述
If you look at http://pulsar.apache.org/docs/en/functions-develop/#serde under the Python tab it says
"In Python, the default SerDe is identity, meaning that the type is serialized as whatever type the producer function returns."
"You can use the IdentitySerde, which leaves the data unchanged. The IdentitySerDe is the default."
This strongly gives the impression that the default `IdentitySerDe1 does not change the message in any way. This is not the case -- it will attempt to convert incoming bytes to one of float, int, string and only leaves it as bytes when they fail. This can result in unexpected conversions (we have had binary data unexpectedly converted to string).
It also attempts the reverse on the function result. Fortunately this does not result in unexpected behaviour, though does lead to muddled/sloppy programming as people are careless with the type of return value.
There are options as how to correct this:
1: Fix the code so the `IdentitySerDe` is just that - it leaves the bytes unchanged. One could then have a `Paddington Bear SerDe` (well intentioned and helpful but tends to get things wrong) which does what the existing `IdentitySerDe` does and also have `FloatSerDe`, `IntSerDe` and `StringSerDe` to cover the other cases reliably.
2: Change the documentation on the `IdentitySerDe` to explain what it really does and its dangers but leave it as the default. Introduce `FloatSerDe`, `IntSerDe`,`StringSerDe` and `BytesSerDe` to cover the cases reliably.
3: Fix the code so the `IdentitySerDe` is just that - it leaves the bytes unchanged. Also have `FloatSerDe`, `IntSerDe` and `StringSerDe` to cover the other cases reliably. Make `StringSerDe` the default on the guess this is the most common use case.
My preference would be option 1, but I suspect the installed code base would need option 2. 3 is a sort of compromise.
貢獻指南
研究方向
從 http://pulsar.apache.org/docs/en/functions-develop/#serde 的 Python SerDe 區段開始,檢查 issue 中描述的目前 IdentitySerDe 行為。首先確認預期的修正是實作、文件,還是兩者都需要;當文件記載的預設行為與實際的位元組和回傳值處理一致,且明確指定的 SerDes 涵蓋預期型別時,即視為完成。
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- python
- 領域
- distributed-systems
- Issue 類型
- 缺陷
- 難度
- 5/5
- 預估耗時
- 一週以上
- 活躍度
- 停滯
- 描述清晰度
- 需要釐清
- 新手友好度
- 20/100