[RFC] Improving public engagement for OpenSearch release process

Summary:

OpenSearch is a fast-growing community-contributed project and currently has around 600+ contributors who actively participate in the growth of the project day-to-day. OpenSearch has been piloting the process of hosting public meetings for some time, where public members can join the meetings and actively participate synchronously. As an open-source project, we aim to establish shared ownership of OpenSearch with the public through an open governance model. We're already publishing OpenSearch release updates via GitHub and Slack. Our next goal is to adopt an open release model, allowing public participation in OpenSearch releases. This proposal seeks feedback on our current release process, roles, and mechanisms to improve public involvement.

Motivation:

OpenSearch currently release major, minor and patch versions on a regular cadence as listed on this release page. The infrastructure to build, test and release artifacts are available at https://build.ci.opensearch.org/. A public release GitHub issue using this release template is created in the opensearch-build repo along with individual component GitHub issues created using this template for all plugins participating in the release. A primary release manager along with secondary release managers corresponding to participating repos are assigned for a release. The primary release manager goes over the individual release issues periodically across the repos and engage the secondary release managers to take appropriate action as needed throughout the release process. A release call is scheduled with necessary stakeholders to complete the release process.

The status of a release is currently tracked in public GitHub issue, but we believe there is additional opportunity for public collaboration during releases. We strongly believe that involving anyone who wants to be involved in the public is an important step an important step to improve the trust and transparency within the public along with getting additional support, feedback from external contributors for closing the gaps, and improvements to the overall release process.

This scope of this RFC doesn’t include standalone OpenSearch client releases that has been already fully automated and self-serviceable by anyone as described in this GitHub issue.

It would be helpful to go over the current roles and responsibilities before diving deep in to the release process.

Roles and responsibilities:

The below roles and responsibilities are currently involved in OpenSearch release cycle process, we want to collaborate with the public to improve it further by adding, modifying roles and responsibilities (as needed).

Role	Responsibilities
Release manager	Responsible for planning, testing, tracking, release, deployment, communication, and risk management. Collaborate with technical and leadership team to finalize a release schedule and plan for OpenSearch Responsible for managing the release lifecycle process that includes, * Scheduling the release * Co-ordinating between teams * Broadcasting release candidate information to gather votes * Execute automated tests * Signing the artifacts * Deploying release artifacts as per the schedule * Completing post release activities Maintain proper coordination between multiple participating teams to update the project related information in publicly accessible platforms
Repository owner	Responsible for assigning a repo level release manager / POC for a specific release Work closely with release manager to identify, remediate possible gaps for a release corresponding to a specific feature on this repo Ensure sanity tests are executed and documented by assigned release manager Help remediate issues found during testing Surface any gaps to release manager in timely manner Provide votes for finalizing release candidate
Maintainers	Perform sanity test on the release candidate Surface any issues, gaps to release manager in timely manner Help remediate issues found during testing Provide votes for finalizing release candidate
Leadership team	The leadership team is responsible for, High-level technical direction of the OpenSearch product The roadmap of the OpenSearch project and releases Creating appropriate Working Groups ( User Feedback, Documentation etc..) to gather the necessary public feedback before making decisions Technical resources (e.g., code repositories, servers) Maintaining maintainers, committers list Call for closing vote / vote to table any issue Go / No-Go vote on product releases

OpenSearch / OpenSearch dashboards Release:

At a high level. OpenSearch release process involves collaboration with multiple maintainers from OpenSearch, OpenSearch dashboards, website, documentation GitHub repositories that are participating in a release. OpenSearch follows de-centralized development model where the development for core engine, plugins happens across different geographical location. The distribution build (OpenSearch, OpenSearch Dashboards) integrates the code across all these repositories for all supported versions via automated CI / CD process to generate, test and publish nightly builds.

These are the pre-requisites that needs to be closed out before the start of release cycle process.

Pre-requisites:

1. Release manager for a specific OpenSearch release is finalized and assigned 28 days before a planned release.
1. Once the release manager is finalized, his details is assigned to the release GitHub issue created in opensearch-build GitHub repository and also broadcasted to public slack channel.
2. Release issue corresponding to major, minor, patch is created using the release template.
1. Release issue is immediately after the OpenSearch core bumps the version to next major, minor, patch version.
3. Release manager is responsible for creating child component issues on all Github repositories participating in a release.
4. Release manager is responsible for releasing both OpenSearch and OpenSearch dashboards.
1. This is subject to change in future as OpenSearch and OpenSearch dashboards are technically independent products with their own release cycle process.
5. Release manager will obtain all required access during release cycle process..
1. This step needs to be completed 5 days before the scheduled preparation phase for a release. Release manager works with the members of opensearch-build team to get all necessary access. They can also reach out to the members via OpenSearch public slack channel (#release)
6. Release manager will co-ordinate with all participating GitHub repo maintainers via OpenSearch public slack channel (#release)
7. Any questions, issues, feedback related to the release cycle, process, mechanism is discussed in OpenSearch public slack channel (#release)

Current release cycle process:

Note: The PR that describes the end to end OpenSearch release process is in review. The information links added to each of the following steps will be part of the opensearch-build GitHub repository once the PR is merged.

• Preparation (14 days from release date) - Owner [Repository owner, leadership team, Release manager ]
• This phase involves preparing for a release that includes assigning a release owner, updating the release cycle dates, finalizing the feature for a release as listed on the roadmap along with creating component release issues across all participating repos in a release.
• Release branch readiness (Not applicable for patch release) (10 days from release date) - Owner [ Repository owner, Release manager ]
• This phase requires that every team participating in a release have their release (candidate) branch created by this date. The distribution build workflow will also start using the release branch to create release candidate instead of .x branches.
• Code Complete / Feature freeze (7 days from release date) [Release manager, Repository owner]
• All maintainers of OpenSearch GitHub repositories targeting a specific release merge their PR’s ( i.e.e vMajor.minor.patch labeled PRs) to release branch branch before release manager creates the release candidate.
• Release candidate creation (6 days from release date) [Release manager]
• The automated build workflows will create release candidates with the finalized commit and publish the docker images and associated distribution artifacts for testing. The release manager will broadcast the release candidate to public for sanity tests.
• Release testing (5 - 2 days from release date) [Release manager, Maintainers, Repository owners]
• Automated integration, BWC and performance testing is kicked off against the release candidate. The release candidate information is also broadcasted to wider group for another round of smoke testing using the standard communication template.
• Go / No-Go voting (1 day from release date) [Release manager, leadership team, Repository owners]
• After all validations and verification, promote the artifacts to production bucket for the final release after getting the sign off from leadership team committee through Go / No-Go voting process.
• Release (Release day!) [Release manager, leadership team, Repository owners ]
• Release the artifacts to production distribution channels, work with website team to update the website and inform the public of the release.
• Post release activities (Release day!) [Release manager]
• This steps ensures that the tags are created for all participating repos, incrementing versions for next patch, minor release along with house cleaning activities. Release manager also collaborates with the website and documentation repo owners to publish the website updates, blog posts and documentation update corresponding to a release.

Note: We have put out a proposal to move away from release train model with defined release date to release window model with a defined release candidate (RC) creation date. We will iterate on “Release candidate” creation along with “Release testing” phase until we finalize the release candidate. Release day will be 2 days after then final RC creation date. Leadership team can also recommend to skip a release version if we are not able to finalize the release candidate within a reasonable time period to avoid cascading effect to releases.

---

Frequently asked questions (FAQ):

How are the release managers finalized for a release?

Release managers are finalized based on volunteer model. A request for release managers for a specific release will be broadcasted in GitHub as we well as OpenSearch Slack channel and the release manager will be selected based on the first come first serve basis.

Note: Release managers has to be a maintainers of Github repositories under OpenSearch project.

Where can I see the finalized features for a specific release ?

You can visit OpenSearch public roadmap to get the comprehensive list of features scheduled to be released in specific OpenSearch version.

Who are the members of the leadership team?

The leadership team is currently seeded by the below list of AWS employees.

• Carl Meadows
• Anandhi Bumstead
• Eli Fisher
• Charlotte Henkle
• Xin Liu
• Nick Nize
• Daniel (dB.) Doubrovkine
• Barani Bikshandi

This is the current list but feel free to provide your feedback to add additional members to the team via OpenSearch public slack channel

Does OpenSearch and OpenSearch dashboards have separate release timeline ?

Currently OpenSearch and OpenSearch dashboards are released at same time (with same version number) as part of OpenSearch release.

How can I participate in release process?

If you are not part of the listed roles, you can still actively participate in release process by sanity testing the release candidate along with providing your feedback, votes for finalizing the release candidate.

How can I become a maintainer of a repository?

Any new members who are interested in becoming maintainers on specific Github repository need to be nominated by the current maintainers through voting process as described here.

---

Next steps:

All the steps listed above in release cycle process are already accessible by the public and the release steps, process are documented in public Github issue. One of the primary focus of this RFC is to surface the release process, roles and responsibilities along with detailed release guide to improve public engagement in OpenSearch release process.

We would like your feedback on,

• Improving the public engagement for OpenSearch release process
• We would like a broader participation in OpenSearch release cycle process. We have updated the release template, SOP and collaboration channels to make the OpenSearch release process more accessible by the public. We need your feedback on any gaps, process technical limitations that would help improve the engagement.
• Formulating process, mechanism to add new roles to release process
• Please provide us feedback to draft a proposal to add, modify roles to OpenSearch release process
• Improving, simplifying the steps involved in current release cycle process
• Help us improve the OpenSearch release process by providing your inputs. We welcome both technical and process inputs.
• Improving the quality of OpenSearch artifacts
• Provide us inputs on improving the quality bar of every OpenSearch releases.
• General comments around the current process including anything that's confusing
• Feel free to provide any other ad-hoc inputs related to release process, public engagement, roles and responsibilities.

We would like get your feedback and suggestions on this RFC to make the release process as seamless as possible. We look forward to collaborating with the public.

relates #166

Start by reading RELEASE_PROCESS_OPENSEARCH.md and the release_template.md and component_release_template.md files in opensearch-build. Review the documented roles, release phases, GitHub issue tracking, and #release Slack coordination. This RFC is complete only when the proposed public engagement model and process changes have been agreed and documented.