JuliaPlots / JuliaPlots/PlotlyJS.jl

Data Transfer Performance

Open
#77 12 comments 2 reactions 0 assignees View on GitHub
perf
Dominant language
Julia
Stars
444
Forks
86
PR merge metrics
No merged PRs in 30d

Description

I promised a little writeup on this so here goes.

Right now marshalling all data to decimal and back is going to be pretty expensive. In an ideal world the marshalling would just be (the equivalent of):

``` julia
xs = Float64[1,2,3]
write(io, xs)
ys = reinterpret(Float64, read(io, 3*4))
```

In fact, you can probably make things even faster by converting the data to `Float16` first. JS can actually support this via the newfangled [typed array](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Typed_arrays) feature. I don't know of an interchange format that supports these kinds of byte arrays directly, but as luck would have it there's a [msgpack implementation](https://github.com/creationix/msgpack-js) which adds it on as an extension.

With a little bit of hacking on MsgPack.jl it should be straightforward to use buffers to transfer numerical data to the frontend, which should make that transfer at least 10dB faster as well as reducing memory footprint.

One snag is that there's no general support for typed arrays in Plotly yet (https://github.com/plotly/plotly.js/issues/860). This will mean converting the buffer to a JS array, which will negate the memory usage benefit but hopefully won't be too much of a performance hit.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.