Dask dataframe chunksize/partioning affects line aggregation
- Dominant language
- Python
- Stars
- 3.6k
- Forks
- 376
- Avg merge
- 4h 32m
- Merged PRs (30d)
- 1
Description
When using a dask dataframe during line aggregation the chunksize affects the result, this is likely because different chunks aren't processed together effectively splitting the path. In this example I create dask dataframes with varying chunksizes and visualize the effect:
```python
%%opts Image [aspect='square' xaxis=None yaxis=None sublabel_format=''] Layout [hspace=0.1 vspace=0.1]
import dask.dataframe as dd
from holoviews.operation.datashader import datashade, aggregate
paths = hv.Path([[(i+x, i*10) for i in range(10)] for x in range(10)])
hv.Layout([aggregate(hv.Curve(dd.from_pandas(paths.dframe(), chunksize=p)),
dynamic=False, width=10, height=10).relabel('Chunksize: %d' % p)
for p in range(1, 11)])
```

You can see that basically every chunksize or number of partitions produces a slightly different result.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.