Azure / Azure/azure-sdk-tools

MCP tools should validate all azure-rest-api-specs PR CI checks before PR creation

Open
#13,735 4 comments 0 reactions 1 assignee Assigned to @mikeharder View on GitHub
AzSDK Tools Agent dev inner loop
Dominant language
C#
Stars
135
Forks
260
Avg merge
3d 1h
Merged PRs (30d)
143

Description

## Problem

The azure-sdk-mcp tools guide users through creating PRs for azure-rest-api-specs, but they don't validate all the CI checks that will run on the PR. This leads to PRs being created that fail CI, requiring additional commits to fix.

## Current Behavior

The \zsdk_run_typespec_validation\ tool only checks that TypeSpec compiles. However, the azure-rest-api-specs PR CI runs many additional checks:

- **TypeSpec Validation (tsv)** - Full validation including generated files committed, formatting, folder structure
- **Swagger LintDiff** - Linting differences in OpenAPI specs
- **Swagger BreakingChange** - Breaking change detection
- **Swagger SemanticValidation** - Semantic validation of OpenAPI specs
- **Swagger ModelValidation** - Model validation
- **Swagger Avocado** - Additional swagger checks
- **SpellCheck** - Spelling validation
- **And more...**

## Expected Behavior

Before creating a PR via \zsdk_create_pull_request\, users should be able to validate that all CI checks will pass. This could be achieved by:

1. **Expanding \zsdk_run_typespec_validation\** to run the full \
px tsv\ validation
2. **Adding new tools** for other validations (lint, breaking changes, etc.)
3. **Creating a comprehensive \zsdk_validate_spec_pr\** tool that runs all checks

## Specific Example

When making TypeSpec changes, the current flow:
1. ✅ \zsdk_run_typespec_validation\ passes
2. ✅ PR created via \zsdk_create_pull_request\
3. ❌ CI fails because generated OpenAPI files weren't committed

The tool should have caught that the generated files needed to be committed before allowing the PR to be created.

## Suggested Solution

Add pre-PR validation that mirrors the CI checks, or at minimum document which manual steps users need to run (like \
px tsv\, \ sp compile\, etc.) before creating a PR.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.