apache / apache/datafusion

[DISCUSS] Release retrospective

Open
#21,034 14 comments 4 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

This topic came up in the recent DataFusion community meeting. There is interest in having a retrospective of our current release practices.

## Background

DataFusion major version 52 ended up with 4 releases and another underway:

- https://github.com/apache/datafusion/issues/20855
- https://github.com/apache/datafusion/issues/20681
- https://github.com/apache/datafusion/issues/20287
- https://github.com/apache/datafusion/issues/19784
- https://github.com/apache/datafusion/issues/18566

At the same time we are trying to release [53.0.0](https://github.com/apache/datafusion/issues/19692). Trying to release patches on past versions while simultaneously releasing a new major version may split maintainer time and focus in undesirable ways.

## Additional references

Discussion about the frequency of releasing: https://github.com/apache/datafusion/issues/16622

We are currently updating the release documentation: https://github.com/apache/datafusion/pull/21001

## Request

This issue is to collect feedback from all community members about the recent release process. The discussion above is where the topic of frequency should happen.

There is a plan to have an in-person retrospective at the next community meeting, March 25. @mbutrovich has volunteered to try capturing notes from that meeting.

For people who want to share thoughts asynchronously or cannot attend the community meeting, please give us your feedback about the recent release process.

- What went well?
- What could be improved?
- What did we learn?
- What should we commit to changing?

## A list of things to ensure nothing gets lost

- [ ] git tag validation requested @coracuity
- [ ] release management doc update
- [ ] put publish crates list in correct order
- [ ] Example for `Download and unpack the official release tarball` like wget from https://dist.apache.org/repos/dist/release/datafusion/datafusion-53.0.0/
- [ ] Followup on https://github.com/apache/datafusion/issues/19692#issuecomment-4114123910 Need to ensure package compiles before doing RC

Contributor guide

Open the contributing guide

Research direction

Start by reading the release-process references in issues 19692 and 16622, the release documentation pull request 21001, and the listed release checklist items. Share feedback using the four questions in this issue, then capture any agreed improvements or meeting notes, including the planned March 25 retrospective outcomes.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, rust
Domain
documentation, release
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.