[DISCUSS] publishing nightly / non-official releases to crates.io (even more frequent releases)
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
DataFusion's mission statement is to be "easy to embed" from https://arrow.apache.org/datafusion/user-guide/introduction.html#why-datafusion
Within the rust ecosystem, publishing to crates.io is the key way to make something easy to embed (other projects can then use it with a single line of `toml`). Thanks to @andygrove and other's work, we now are publishing official versions about once a month ❤️ (e.g. #4776 )
However, releasing even every month is still slower than optimal, as sometimes there are changes / bugs to the interfaces which aren't caught until after the release (e.g. most recently necessitating a patch release https://github.com/apache/arrow-datafusion/issues/4948)
Many active users of DataFusion today still use a fork or work off the master branch directly but this is not always possible for all projects as it is somewhat challenging.
Having more frequently published versions on crates.io would help others integrate datafusion more quickly between major official releases
**Describe the solution you'd like**
I propose publishing "nightly" releases of DataFusion to crates.io, much like the rust compiler itself.
These releases would be clearly labeled as "NON OFFICIAL RELEASES" perhaps with versions like `17.0.0.dev-2023-01-22
**Describe alternatives you've considered**
We could also try to run the official release process more frequently (every 2 weeks perhaps) -- given the voting process takes at least three days it will still be limited in how quickly it can be done
**Additional context**
See https://github.com/apache/arrow-datafusion/issues/2327 for prior discussion
Contributor guide
Research direction
Start by reading the prior discussion in issue #2327 and the current official release process referenced through #4776. Determine whether crates.io supports the proposed nightly versioning and publication workflow, and document an agreed approach that clearly distinguishes non-official releases from official versions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- release
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100