holoviz / holoviz/hvplot

How to optimize rasterize with many unique categories

Open
#1,259 0 comments 0 reactions 0 assignees View on GitHub
api: datashader
Dominant language
Python
Stars
1.4k
Forks
124
Avg merge
1d 18h
Merged PRs (30d)
1

Description

I'm trying to do the following, but it takes forever. If I take the `.head(100)`, it works.
```python
import pandas as pd
import datashader as ds
import hvplot.pandas

NETWORKS_URL = "https://mesonet.agron.iastate.edu/sites/networks.php?network=_ALL_&format=csv&nohtml=on"

networks_df = pd.read_csv(NETWORKS_URL)
networks_df.hvplot.scatter(
"lon",
"lat",
legend=False,
cmap="category10",
rasterize=True,
aggregator=ds.count_cat('iem_network'),
hover_cols=["stid", "station_name", 'iem_network'],
)
```

Another way is just using `color` instead and it's done in a flash.
```python
import pandas as pd
import datashader as ds
import hvplot.pandas

NETWORKS_URL = "https://mesonet.agron.iastate.edu/sites/networks.php?network=_ALL_&format=csv&nohtml=on"

networks_df = pd.read_csv(NETWORKS_URL)
networks_df.hvplot.scatter(
"lon",
"lat",
legend=False,
cmap="category10",
color="iem_network",
alpha=0.25,
size=5,
hover_cols=["stid", "station_name", 'iem_network'],
)
```
image

I think `rasterize` does something with `by` internally(?) and it's trying to overlay each one, but there's too many categories.

Related: https://discourse.holoviz.org/t/use-color-instead-of-by-when-possible/6784/1

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.