JuliaPlots / JuliaPlots/PlotlyJS.jl
Data Transfer Performance
- Dominant language
- Julia
- Stars
- 444
- Forks
- 86
- PR merge metrics
- No merged PRs in 30d
Description
I promised a little writeup on this so here goes.
Right now marshalling all data to decimal and back is going to be pretty expensive. In an ideal world the marshalling would just be (the equivalent of):
``` julia
xs = Float64[1,2,3]
write(io, xs)
ys = reinterpret(Float64, read(io, 3*4))
```
In fact, you can probably make things even faster by converting the data to `Float16` first. JS can actually support this via the newfangled [typed array](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Typed_arrays) feature. I don't know of an interchange format that supports these kinds of byte arrays directly, but as luck would have it there's a [msgpack implementation](https://github.com/creationix/msgpack-js) which adds it on as an extension.
With a little bit of hacking on MsgPack.jl it should be straightforward to use buffers to transfer numerical data to the frontend, which should make that transfer at least 10dB faster as well as reducing memory footprint.
One snag is that there's no general support for typed arrays in Plotly yet (https://github.com/plotly/plotly.js/issues/860). This will mean converting the buffer to a JS array, which will negate the memory usage benefit but hopefully won't be too much of a performance hit.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.