Calling datashader from R
- Dominant language
- Python
- Stars
- 3.6k
- Forks
- 376
- Avg merge
- 4h 32m
- Merged PRs (30d)
- 1
Description
The principles of datashader are applicable to any language, and in particular there are many data scientists who use R who could benefit from an implementation that is easily accessible from R.
Datashader consists of a relatively small number of lines of Python that in principle could be translated into R, but datashader gets its speed from [Numba](http://numba.pydata.org) (which compiles Python down to bare machine code), and its ability to handle out-of-core and distributed computation from [Dask](http://dask.pydata.org). LIbraries with those capabilities are not yet available in the R ecosystem, and so any reasonable R implementation of datashader would presumably be too slow and limited to be viable.
So, is there a convenient way that we could make datashader as already written be easily invoked from R? The [Apache Arrow](https://arrow.apache.org) project gives one possibility, by providing an in-memory format for columnar data that both R and Python could access. Related work from the [Feather](https://github.com/wesm/feather) project has already produced libraries for both languages, which could be a good first step by making fast to transfer data on disk between the two languages.
So, the first step is to set up a basic pathway for taking an R data structure, sending the data to datashader in Python, collecting the resulting image or aggregate array, and displaying it in an R-based plot. From that starting point, we can decide what the major bottlenecks are and whether they are feasible to address. Help from interested R users will make this happen much quicker!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.