`PlotlyJSONEncoder` always casts values to float64 due to using `tolist()`
還沒有人認領這個 Issue。
- 主要語言
- Python
- 星號
- 18.8k
- 分支
- 2.8k
- 平均合併
- 16 小時 26 分鐘
- 30 天內合併 PR
- 21
描述
Regarding the numpy floating point precision and that PlotlyJSONEncoder always casts those to float64 due to using tolist()...
This had always bugged me, as it resulted in much larger exports (i.e. html / ipynb file sizes) than necessary (when float16 or float32 is sufficient) and affected not only coordinate data, but also marker sizes, meta info, etc.
Just in case the plotly.py devs or others are interested: I had found a way to avoid this number inflation by modifying (& monkey patching) the encode_as_list method:
@staticmethod
def encode_as_list_patch(obj):
"""Attempt to use `tolist` method to convert to normal Python list."""
if hasattr(obj, "tolist"):
numpy = get_module("numpy")
try:
if isinstance(obj, numpy.ndarray) \
and obj.dtype == numpy.float32 or obj.dtype == numpy.float16 \
and obj.flags.contiguous:
return [float('%s' % x) for x in obj]
except AttributeError:
raise NotEncodable
return obj.tolist()
else:
raise NotEncodable
It's about 30-50x slower than .tolist(), but - being in the order of a few μs - still much faster than the json encoding, with the benefit of ~3x smaller exports.
I always wanted to report this, and this PR revived the topic. Could this be relevant for a new issue (especially since orjson will not become the default)?
FYI: for reference, a quick search revealed that a patch of encode_as_list was already suggested before: https://github.com/plotly/plotly.py/issues/1842#issuecomment-549401190, in the context of treating inf & NaN, which got brought up again in https://github.com/plotly/plotly.py/pull/2880#issuecomment-726860782.
Originally posted by @mherrmann3 in https://github.com/plotly/plotly.py/issues/2955#issuecomment-856651213
貢獻指南
從這裡開始
- 先讀完整個 Issue,再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫,在一個分支上完成修改。
- 送出 Pull Request,並在描述裡引用這個 Issue 編號。
研究方向
從 packages/python/plotly/_plotly_utils/utils.py 中的 PlotlyJSONEncoder.encode_as_list 開始,然後查看關於 NumPy float16 和 float32 處理方式的相關討論。確定在不改變其他值的情況下,保留它們的精度會如何影響序列化和匯出大小;完成的標準是,相關 NumPy 型別的行為得到驗證,且現有的編碼行為保持不變。
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- numpy, python
- 領域
- data-visualization
- Issue 類型
- 缺陷
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 活躍度
- 停滯
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100