dask in pyscript
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
For those that didn't hear the news, [pyscript](https://pyscript.net/) is a new project from Anaconda, to run a complete CPython runtime in the browser via wasm. I will post the pycon keynote when it becomes available. There are already impressive demos interacting with existing pydata tools (numpy, pandas) as well as hooks into JS display tools (d3...) *without any server at all*.
One thing pyscript does not have is a good data story, because the browser environment doesn't support (TCP) "sockets" , only HTTP(s) as provided by the browser runtime, and limited by CORS. That means that fsspec has a real hard time doing anything useful.
On the other hand, dask is able to talk to a distributed cluster over websockets - this already works today. My question is, is there any interest in pushing dask forward as an in-browser, async-mode client? Interestingly, this would enable full fsspec operation by doing all operations via a worker. From the point of view of Coiled (or others that run a remote scheduler), it would get around the tricky/jhub issue of where to run the python kernel.
Progress would require a complex build chain to wasm-ify dask's dependencies, but I think that so long as we steer away from explicit socket-level stuff by relying on websockets, it should be doable. (noting that the current python websocket stack relies on python's builtin httplib/sockets, so would need rewriting to use the browser's internal JS-facing interface).
Contributor guide
Assessment
This issue has not been assessed yet.