holoviz / holoviz/datashader

Adding a NumPy backend

Open
#289 1 comment 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
3.6k
Forks
376
Avg merge
4h 32m
Merged PRs (30d)
1

Description

Data often starts out as NumPy arrays rather than Dask or Pandas dataframes, and it could be useful to work directly with such arrays (see #283). I briefly investigated how much work it would be to use an OrderedDict of NumPy arrays of the same size, by searching the code for "df" (maybe not reliable, but I think it covers everywhere the dataframe is accessed). Findings:

1. Some usages of `df` that would just work already for the OrderedDict case:
```
df[self.x].min()
df[self.x].max()
```

2. Some accesses that are already just skipping to the underlying NumPy array anyway, so we could add a conditional or make a simple wrapper function to do the right thing in both cases:
```
df[y_name].values
```

3. Categorical data would require some extra work (we currently access `df[self.column].cat.codes.values`); those values would need to be stored somewhere.

4. We already have multi-dispatch to handle the actual computational bits, and we'd need an equivalent to datashader/pandas.py, which isn't much code but is fairly mysterious.

So it would be a bit of a project, but I think it would be feasible to support NumPy using arrays named in OrderedDicts. Something for the wishlist, or a good project for a motivated contributor who wants to really understand how everything fits together in datashader...

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.