TimelyDataflow / TimelyDataflow/differential-dataflow
Suggestion: WebAssembly support
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 3k
- Forks
- 211
- Avg merge
- 10h 42m
- Merged PRs (30d)
- 34
Description
Hi,
I was interested in seeing if I could run differential-dataflow on the web by compiling to WebAssembly using wasm-pack and it worked out really well! However it required some hacks changes to differential-dataflow and timely-dataflow, specifically:
- For both packages, replacing
std::time::Instantwithweb_time::Instant- crate: web-time This is because there is no "current time" implemented for thewasm32-unknown-unknowncompilation target - so we have to hook it up to the browser's Performance.now() function. In all other contexts this uses the regularstd::time::Instantunder the hood so shouldn't make any difference. - Differential dataflow has a 63 bit shift
(1 << 63)which needs to be(1 << 31)in 32-bit WebAssembly.
The changes required are:
- Timely dataflow: https://github.com/TimelyDataflow/timely-dataflow/compare/master...oli-w:timely-dataflow:master
- Differential dataflow: https://github.com/TimelyDataflow/differential-dataflow/compare/master...oli-w:differential-dataflow:master
I was wondering if you would be interested in incorporating this or would prefer to keep this kind of change as a fork?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the linked timely-dataflow and differential-dataflow compare branches, then check the wasm-pack approach and the wasm32-unknown-unknown constraints described in the issue. Done means reaching a maintainer decision and, if accepted, incorporating equivalent WebAssembly support in both packages while preserving non-WebAssembly behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, wasm
- Domain
- build-system, data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100