plotly / plotly/plotly.py

`PlotlyJSONEncoder` always casts values to float64 due to using `tolist()`

未关闭
#3,232 5 条评论 1 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

feature P3
主要语言
Python
星标
18.8k
派生
2.8k
平均合并
16 小时 26 分钟
30 天内合并 PR
21

描述

Regarding the numpy floating point precision and that PlotlyJSONEncoder always casts those to float64 due to using tolist()...

This had always bugged me, as it resulted in much larger exports (i.e. html / ipynb file sizes) than necessary (when float16 or float32 is sufficient) and affected not only coordinate data, but also marker sizes, meta info, etc.

Just in case the plotly.py devs or others are interested: I had found a way to avoid this number inflation by modifying (& monkey patching) the encode_as_list method:

@staticmethod
def encode_as_list_patch(obj):
    """Attempt to use `tolist` method to convert to normal Python list."""
    if hasattr(obj, "tolist"):

        numpy = get_module("numpy")
        try:
            if isinstance(obj, numpy.ndarray) \
               and obj.dtype == numpy.float32 or obj.dtype == numpy.float16 \
               and obj.flags.contiguous:
                return [float('%s' % x) for x in obj]
        except AttributeError:
            raise NotEncodable

        return obj.tolist()
    else:
        raise NotEncodable

It's about 30-50x slower than .tolist(), but - being in the order of a few μs - still much faster than the json encoding, with the benefit of ~3x smaller exports.

I always wanted to report this, and this PR revived the topic. Could this be relevant for a new issue (especially since orjson will not become the default)?

FYI: for reference, a quick search revealed that a patch of encode_as_list was already suggested before: https://github.com/plotly/plotly.py/issues/1842#issuecomment-549401190, in the context of treating inf & NaN, which got brought up again in https://github.com/plotly/plotly.py/pull/2880#issuecomment-726860782.

Originally posted by @mherrmann3 in https://github.com/plotly/plotly.py/issues/2955#issuecomment-856651213

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

从 packages/python/plotly/_plotly_utils/utils.py 中的 PlotlyJSONEncoder.encode_as_list 开始,然后查看关于 NumPy float16 和 float32 处理方式的相关讨论。确定在不改变其他值的情况下,保留它们的精度会如何影响序列化和导出大小;完成的标准是,相关 NumPy 类型的行为得到验证,并且现有的编码行为保持不变。

由索引模型根据 Issue 内容生成。

评估

技术栈
numpy, python
领域
data-visualization
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
35/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。