How to optimize rasterize with many unique categories
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 124
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 1
Description
I'm trying to do the following, but it takes forever. If I take the `.head(100)`, it works.
```python
import pandas as pd
import datashader as ds
import hvplot.pandas
NETWORKS_URL = "https://mesonet.agron.iastate.edu/sites/networks.php?network=_ALL_&format=csv&nohtml=on"
networks_df = pd.read_csv(NETWORKS_URL)
networks_df.hvplot.scatter(
"lon",
"lat",
legend=False,
cmap="category10",
rasterize=True,
aggregator=ds.count_cat('iem_network'),
hover_cols=["stid", "station_name", 'iem_network'],
)
```
Another way is just using `color` instead and it's done in a flash.
```python
import pandas as pd
import datashader as ds
import hvplot.pandas
NETWORKS_URL = "https://mesonet.agron.iastate.edu/sites/networks.php?network=_ALL_&format=csv&nohtml=on"
networks_df = pd.read_csv(NETWORKS_URL)
networks_df.hvplot.scatter(
"lon",
"lat",
legend=False,
cmap="category10",
color="iem_network",
alpha=0.25,
size=5,
hover_cols=["stid", "station_name", 'iem_network'],
)
```
I think `rasterize` does something with `by` internally(?) and it's trying to overlay each one, but there's too many categories.
Related: https://discourse.holoviz.org/t/use-color-instead-of-by-when-possible/6784/1
Contributor guide
Assessment
This issue has not been assessed yet.