plotly / plotly/plotly.py

significant speedup for "to_scalar_or_list"

Open
#2,938 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

P3 performance
Dominant language
Python
Stars
18.8k
Forks
2.8k
Avg merge
16h 26m
Merged PRs (30d)
21

Description

hello,

While investigating a slowness in plotly, I have stumbled upon the to_scalar_or_list function (https://github.com/plotly/plotly.py/blob/abd86092e048d5c8b02da65123824b75e4311838/packages/python/plotly/_plotly_utils/basevalidators.py#L30) that was taking much time.
After some tinkering, I came with the two following changes that vastly improves the performance:

So at the end, it looks like

np = get_module("numpy", should_load=False)
pd = get_module("pandas", should_load=False)
# Utility functions
# -----------------
def to_scalar_or_list(v):
    # Handle the case where 'v' is a non-native scalar-like type,
    # such as numpy.float32. Without this case, the object might be
    # considered numpy-convertable and therefore promoted to a
    # 0-dimensional array, but we instead want it converted to a
    # Python native scalar type ('float' in the example above).
    # We explicitly check if is has the 'item' method, which conventionally
    # converts these types to native scalars.

    # check first for the simple case
    if isinstance(v,(int,float,str)):
        return v
    if np and np.isscalar(v) and hasattr(v, "item"):
        return v.item()
    if isinstance(v, (list, tuple)):
        return [to_scalar_or_list(e) for e in v]
    elif np and isinstance(v, np.ndarray):
        if v.ndim == 0:
            return v.item()
        return [to_scalar_or_list(e) for e in v]
    elif pd and isinstance(v, (pd.Series, pd.Index)):
        return [to_scalar_or_list(e) for e in v]
    elif is_numpy_convertable(v):
        return to_scalar_or_list(np.array(v))
    else:
        return v

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in packages/python/plotly/_plotly_utils/basevalidators.py at to_scalar_or_list and inspect the nearby get_module and is_numpy_convertable usage. Compare behavior and performance for native scalars, lists or tuples, NumPy arrays, and pandas Series or Index values. Done means the proposed cases retain their existing conversions while avoiding unnecessary repeated work.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, pandas, python
Domain
performance
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.