pydata / pydata/xarray

Call .compute() in all plot methods?

Open
#5,589 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug topic-plotting
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

I noticed what I think might be a performance bug: should .compute() be called on the input data in all the plotting methods (e.g. plot.pcolormesh()) like it is in .plot() here https://github.com/pydata/xarray/blob/bf27e2c1fd81f5e0afe1ef91c13f651db54b44bb/xarray/plot/plot.py#L166 See also discussion in https://github.com/pydata/xarray/issues/2390.

I was making plots from a large dataset of a quantity that is the output of quite a bit of computation. A script which made an animation of the full time-series (a couple of thousand time points) actually ran significantly faster than a script that made pcolormesh plots of just 3 time points (~2hrs compared to ~5hrs). The difference I can think of is that the animation script called .values before the animation function, but the plotting script called xarray.plot.pcolormesh() without calling .values/.load()/.compute() first. A modified version of the script that calls .load() before any plot calls reduced the run time to 30 mins even though I plotted 18 time points, not just 3.

2d plots might all be covered by adding a darray = darray.compute() call in newplotfunc()?https://github.com/pydata/xarray/blob/bf27e2c1fd81f5e0afe1ef91c13f651db54b44bb/xarray/plot/plot.py#L609
I guess the 1d plot functions would all need to be modified individually.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in xarray/plot/plot.py, comparing the existing .plot() call near line 166 with newplotfunc() near line 609 and the individual 1D plotting methods. Investigate whether lazy inputs are recomputed across pcolormesh() and related methods, using the reported large-dataset plotting behavior as a benchmark. Done means the appropriate plotting methods handle computation consistently without the reported repeated-work slowdown.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-visualization, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.