apache / apache/datafusion-ray
[EPIC] DataFusion on Ray 0.1.0 release
- Dominant language
- Python
- Stars
- 230
- Forks
- 29
- PR merge metrics
- No merged PRs in 30d
Description
## Manage donation
- [x] Propose donation of Ray SQL into Apache DataFusion - https://github.com/apache/datafusion-python/issues/872
- [x] Vote to accept donation - https://lists.apache.org/thread/2cff6k3jtzg8ob1x1qz22150swwnkqsn
- [x] Complete the IP clearance process - https://lists.apache.org/thread/hytm7fl0z3c24sdhf9wbdvnpk2o5o9or
- [x] Merge https://github.com/apache/datafusion-ray/pull/1
- [x] Update the README in the ray-sql repo to state that the project has moved, and archive the repo
## Initial tasks once the donation has been accepted
- [x] Update packaging and documentation to reflect the new project name and purpose (it is no longer a research project, for example)
- [x] Upgrade to use DataFusion 41 (based on https://github.com/datafusion-contrib/ray-sql/pull/51)
- [x] Get the Rust unit tests in `planner.rs` running in CI and update the expected plans under `testdata/expected-plans`
- [x] Manually test on a distributed cluster to confirm that everything is working - https://github.com/apache/datafusion-ray/pull/20
## Benchmarking
- [x] Add documentation for running benchmarks
- [ ] Automate running benchmarks against PRs using compute infrastructure provided by @andygrove
- [ ] Update performance charts in README
## Set up release process
- [x] Add scripts for creating and publishing source releases
- [ ] Add CI scripts for building Python wheels
- [ ] Add CI scripts for building Docker images
- [ ] Set up RAT checks in CI
## First Release
- [ ] Release datafusion-ray 0.1.0
- [ ] Write a blog post to announce the new project - Google doc: https://docs.google.com/document/d/18Yd10y9fhIKiAfUPifQk5BSFytOGV0tcbmnK2Xy7_Ro/edit?usp=sharing
Contributor guide
Assessment
This issue has not been assessed yet.