holoviz / holoviz/datashader

Tell datashader to use a specific color for NaNs in categorical data

Open
#1,019 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
3.6k
Forks
376
Avg merge
4h 32m
Merged PRs (30d)
1

Description

I have very large dataset that I cannot plot directly using holoviews. I want to make a scatterplot with categorial data. Unfortunately my data is very sparse and many points have NA as category. I would like to make these points gray. Is there any way to make datashader know what I want to do?

I show you the way I do it now (as more or less proposed in https://holoviews.org/user_guide/Large_Data.html ). I provide you an example:

```
import numpy as np
import pandas as pd
import holoviews as hv
hv.extension('bokeh')
import datashader as ds
from datashader.colors import Sets1to3
from holoviews.operation.datashader import datashade,dynspread

raw_data = [('Alice', 60, 'London', 5) ,
('Bob', 14, 'Delhi' , 7) ,
('Charlie', 66, np.NaN, 11) ,
('Dave', np.NaN,'Delhi' , 15) ,
('Eveline', 33, 'Delhi' , 4) ,
('Fred', 32, 'New York', np.NaN ),
('George', 95, 'Paris', 11)
]
# Create a DataFrame object
df = pd.DataFrame(raw_data, columns=['Name', 'Age', 'City', 'Experience'])
df['City']=pd.Categorical(df['City'])

x='Age'
y='Experience'
color='City'
cats=df[color].cat.categories

# Make dummy-points (currently the only way to make a legend: https://holoviews.org/user_guide/Large_Data.html)
for cat in cats:
#Just to make clear how many points of a given category we have
print(cat,((df[color]==cat)&(df[x].notnull())&(df[y].notnull())).sum())
color_key=[(name,color) for name, color in zip(cats,Sets1to3)]
color_points = hv.NdOverlay({n: hv.Points([0,0], label=str(n)).opts(color=c,size=0) for n,c in color_key})

# Create the plot with datashader
points=hv.Points(df, [x, y],label="%s vs %s" % (x, y),)
datashaded=datashade(points,aggregator=ds.by(color)).opts(width=800, height=480)

(dynspread(datashaded)*color_points).opts(legend_position='right')

```

It produces the following picture:
![bug_report_image](https://user-images.githubusercontent.com/83504577/130501465-76ba8d78-42d0-4656-8a50-ab5fa36b49f3.png)

Although there is just one person from Paris you see that the NA-person (Charlie) is also printed in purple, the color for Paris. Is there a way to make the dot gray? I have tried many plots and it seems like the NAs always take the color of the last item in the legend.

It would be nice to provide the possiblilty to give a parameter like ```NA_color='gray'``` to the ```datashade```-method. Also an option for not plotting NA-category-points at all would be nice with the same kind of interface. But that is less important.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the provided Python example using holoviews.operation.datashader.datashade with datashader.ds.by(color), focusing on the categorical NaN case. The change is done when NaN-category points consistently use a caller-selected gray color, with an option to omit them if supported by the chosen design.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, pandas, python
Domain
data-visualization
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.