Status of dask support in pyresample?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 385
- Forks
- 102
- Avg merge
- 4d 2h
- Merged PRs (30d)
- 9
Description
The latest pyresample docs state:
Interfaces to XArray objects (including dask array support) are provided in separate Resampler class interfaces and are in active development.
However, there is no further mention of xarray / dask support in the rest of the docs. There seem to be a few dask issues (e.g. #148), but I could not ascertain the status of xarray / dask support based on browsing the docs and the repo.
Could you clarify where things stand? My use case is that I would like to use pyresample lazily on dask arrays, where the data is chunked contiguously in space but has many samples in time. Here's what I have tried, representing each timestep as a different channel:
import numpy as np
import dask.array as da
from pyresample import image, geometry
area_def = geometry.AreaDefinition('areaD', 'Europe (3km, HRV, VTC)', 'areaD',
{'a': '6378144.0', 'b': '6356759.0',
'lat_0': '50.00', 'lat_ts': '50.00',
'lon_0': '8.00', 'proj': 'stere'},
800, 800,
[-1370912.72, -909968.64,
1029087.28, 1490031.36])
msg_area = geometry.AreaDefinition('msg_full', 'Full globe MSG image 0 degrees',
'msg_full',
{'a': '6378169.0', 'b': '6356584.0',
'h': '35785831.0', 'lon_0': '0',
'proj': 'geos'},
3712, 3712,
[-5568742.4, -5568742.4,
5568742.4, 5568742.4])
# Here I have 10 "timesteps" (pyresample calls these "channels")
# The channels are the final axis (axis=2) of the array
# works if I use numpy
# data = np.random.rand(3712, 3712, 10)
data = da.random.random((3712, 3712, 10), chunks=(3712, 3712, 1))
msg_con_nn = image.ImageContainerNearest(data, msg_area, radius_of_influence=50000)
msg_con_nn.resample(area_def)
I would expect this to lazily return a dask array with the same chunk structure as the input array, with resampling performed on demand as the chunks are loaded. Instead I hit an error when creating the ImageContainerNearest object:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-9-d3a77de18e5c> in <module>
1 #data = np.random.rand(3712, 3712, 10)
2 data = da.random.random((3712, 3712, 10), chunks=(3712, 3712, 1))
----> 3 msg_con_nn = image.ImageContainerNearest(data, msg_area, radius_of_influence=50000)
/srv/conda/envs/notebook/lib/python3.7/site-packages/pyresample/image.py in __init__(self, image_data, geo_def, radius_of_influence, epsilon, fill_value, reduce_data, nprocs, segments)
255 super(ImageContainerNearest, self).__init__(image_data, geo_def,
256 fill_value=fill_value,
--> 257 nprocs=nprocs)
258 self.radius_of_influence = radius_of_influence
259 self.epsilon = epsilon
/srv/conda/envs/notebook/lib/python3.7/site-packages/pyresample/image.py in __init__(self, image_data, geo_def, fill_value, nprocs)
59 geo_def = geo_def.freeze()
60 if not isinstance(image_data, (np.ndarray, np.ma.core.MaskedArray)):
---> 61 raise TypeError('image_data must be either an ndarray'
62 ' or a masked array')
63 elif ((image_data.ndim > geo_def.ndim + 1) or
TypeError: image_data must be either an ndarray or a masked array
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the latest pyresample documentation and the image.ImageContainerNearest entry point, then review the dask-related context in issue #148 and the reported dask.array example. Clarify the current xarray/dask support and document the supported behavior or limitations, including whether lazy resampling is expected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100