microsoft / microsoft/SynapseML

chore: automate release step 1.6 - queue ADO pipeline 17563 for each release tag

Open
#2,641 1 comment 0 reactions 0 assignees View on GitHub
triage
Dominant language
Scala
Stars
5.2k
Forks
868
Avg merge
22h 9m
Merged PRs (30d)
45

Description

Follow-up to #2540, which automates Release Guide steps 1.4–1.5 (create the release tags, open the spark rebase PRs, tag the merged spark branches).

The next manual step is 1.6: for each of the tags that #2540 now creates, queue ADO pipeline `17563` (`msdata/A365`) against that tag and wait for it to complete. That is one queue-and-poll per tag — `vX.Y.Z`, `-python3.11`, `-spark3.5`, `-spark4.0`, `-spark4.1` — done by hand today.

This looks automatable from GitHub Actions. `secrets.ADO_PERSONAL_ACCESS_TOKEN` already exists in this repo and is used by `.github/workflows/ado-integration.yml`, so the plumbing is there.

Before doing it, two things need checking:

1. **Token scope.** Queueing an official release build is a materially bigger grant than what `ado-integration.yml` does today. Whether the existing PAT already has `Build (read & execute)` on `A365`, and whether we *want* a GitHub-held secret to be able to queue official builds, is a security call, not a mechanical one.
2. **Failure semantics.** A failed release build should stop the release loudly. Needs to fail the workflow and not proceed to the remaining tags.

Explicitly *not* in scope for this or any follow-up, because they cannot be automated from a public repo:

- **1.6a ESRP signing approval** — `EsrpRelease@9` under an AME-tenant managed identity, interactive approval from a named list on a SAW machine.
- **1.6b Maven Central / PyPI publish** — already automated inside `pipeline.yaml`; the credentials live in ADO KeyVault `mmlspark-keys` and must not be mirrored into GitHub.
- **Steps 2–5** — SynapseML-Internal, Publish-Official, BBC-VHD/White Glove. Different ADO repos and pipelines, one of them gated on a 1–2 week human approval board.

So this issue is the *last* release step that can move to GitHub Actions. After it, the release is a human gate by design.

Contributor guide

Open the contributing guide

Research direction

Start with .github/workflows/ado-integration.yml and inspect how ADO_PERSONAL_ACCESS_TOKEN is used, then review pipeline.yaml and issue #2540 for the release-tag flow. Confirm the PAT scope and security decision before implementing queue-and-poll behavior for pipeline 17563. Done means each listed tag is processed in order, failures stop the workflow, and successful completion is visible in GitHub Actions.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, github-actions
Domain
ci-cd, devops, release
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.