Azure / Azure/azure-sdk-tools

[Tracking] SDK PR Release Readiness: Gaps and Known Issues

Open
#15,705 5 comments 0 reactions 1 assignee Claimed by @raych1 View on GitHub
AzSDK Tools Agent Epic needs-triage
Dominant language
C#
Stars
135
Forks
260
Avg merge
3d 1h
Merged PRs (30d)
144

Description

# [Tracking] SDK PR Release Readiness: Gaps and Known Issues

## Context
This parent issue consolidates SDK PR readiness gaps and links to existing issues in the Azure SDK release workflow, covering validation, tooling, and pipelines.
---

## 🔍 Gaps Identified
- [x] **SDK PR Information** - The title and description are not updated when the SDK PR is re-generated.
- [x] Update the title and description if the SDK PR already exists during PR creation.
- [ ] **Late spec validation** – Spec issues (namespace approval, API version, spec branch) are only detected during SDK PR review instead of earlier stages
- [x] Namespace approval: how can we validate the status automatically?
- [ ] API version: is this already validated when service team start the release?
- [x] Spec branch and commit: leverage the existed `verify-restapi-spec-location` check for validation.

- [ ] **Excessive manual effort in SDK PR readiness** – Auto-generated SDK PRs are not fully ready for review and require manual fixes
- [ ] Linter failures (especially samples)
- [x] Recorded test failures (due to API/version changes). Propose adding a feature in the `test-proxy` to sanitize the `api-version`.
- [x] Merge conflicts (copilot can help directly in the PR).
(According to Weidong, conflicts mostly happen when the SDK PR is not merged in time.)
- [x] Breaking changes mitigation (This will be covered in the new SDK breaking change detection tool)
- [ ] What are the other types of manual fixes?

- [ ] **.NET specific gaps**
- [ ] Missing GA test gating – Required tests for GA releases are not strictly enforced, allowing gaps in release quality
- [x] Enforce the code coverage checks? (We currently lean toward not doing this, based on feedback from Michael: >any requirement we add to code coverage will result in synthetic tests that traverse every line of code but don't meaningfully test anything )
- [ ] Redundant APIView requirement – APIView approval overlaps with SDK PR review and may not provide additional value

- [ ] **PR approval and merge**
- [ ] Enable auto-approve for SDK PR
- [x] ~~Enable auto-merge for SDK PR~~ (We decided not to enable this because the added value is limited)
- [ ] **Release pipeline provisioning delay** – Pipelines for newly onboarded RPs are created asynchronously (overnight/weekend), delaying readiness
- [x] Propose enabling CI trigger for the `prepare-pipelines` pipeline to eliminate the need for manual runs

- [ ] **Manual release pipeline triggering** – Release depends on service teams remembering target dates; no automatic fallback mechanism when missed
- [ ] Trigger the release pipeline automatically when the SDK PR is merged?
- [ ] **Pipeline failures block release**
- [ ] Failures caused by network isolation policies and CFS security requirements (the majority of the required work has already been done, though will likely be a long tail as NI policies continue to roll out)
- [x] ESRP publishing failures
- [x] NPM account token rotation (performed once as a security response following a supply chain attack, and once as part of routine maintenance. This is fully managed by the ESRP team, and there is no mitigation action available on our side.)
- [x] Maven fails to render the package even though ESRP publish operation succeeds (This happened once and appears to have been a maven central site issue. we will need escalation to the maven support team or publishing a new version)

---

## 🔗 Related Work / Existing Issues

- [Simplify Mgmt SDK release workflow](https://github.com/Azure/azure-sdk-tools/issues/14651)
- [Add validation before PR review](https://github.com/Azure/azure-sdk-tools/issues/12831)
- [.NET specific gaps](https://github.com/Azure/azure-sdk-tools/issues/15609)
- [Fix generation pipeline problems](https://github.com/Azure/azure-sdk-tools/issues/15537)

---

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.