Azure / Azure/azure-sdk-tools

Support check-enforcer in the centralized azsdkauto webapp

Open
#16,841 0 comments 0 reactions 0 assignees View on GitHub
Central-EngSys
Dominant language
C#
Stars
135
Forks
260
Avg merge
3d 1h
Merged PRs (30d)
144

Description

## Motivation

The current public GitHub Action listens to slash commands without adequately authorizing the commenter, causing recurring MSRC reports.
The implementation cannot simply move to a private repository while public workflows directly reference its source.

## Background

`azure-sdk-tools` still relies on Check Enforcer because its tools do not share a single unified pull-request pipeline. Other Azure SDK repositories have retired Check Enforcer, but disabling it here without a replacement would remove required-check coverage.

The current [`event.yml`](https://github.com/Azure/azure-sdk-tools/blob/main/.github/workflows/event.yml) listens for `check_suite.completed` and `issue_comment.created`, then runs `azure/azure-sdk-actions@main` with repository-scoped permissions. Keeping this public action and its slash-command listener has also generated recurring security concerns, particularly because comments can invoke the listener without an authorization gate.

Port the required behavior into the centralized `azsdkauto` GitHub App/web app. This lets `azure-sdk-tools` preserve its required-check behavior while removing the public workflow/action dependency and allowing the old implementation to move out of the public source path.

This follows the retirement work in #15160, which identified `azure-sdk-tools` as a late transitioner pending an app-based replacement.

## Scope

- Handle `check_suite.completed` events and publish the same Check Enforcer status/check result used by branch protection today.
- Handle the supported PR comment commands: `/check-enforcer help`, `/check-enforcer evaluate`, and `/check-enforcer override`.
- Authorize commands that can evaluate or override status. At minimum, require repository write/admin permission; document and test the final policy.
- Preserve current Azure Pipelines filtering and success/failure behavior needed by `azure-sdk-tools`.
- Reuse/port the existing Check Enforcer unit tests and webhook payloads from `azure-sdk-actions` as behavior-parity tests.
- Add operational logging sufficient to diagnose ignored events, authorization failures, and status update failures.
- Enable the implementation for `Azure/azure-sdk-tools` as an early production validation of `azsdkauto`.

## Acceptance criteria

- [ ] `azsdkauto` receives the required GitHub events and has only the permissions needed to read checks/PRs and write statuses/comments.
- [ ] For representative existing payloads, app-generated status results match the current Check Enforcer behavior.
- [ ] `help`, `evaluate`, and `override` have automated coverage, including authorized and unauthorized commenters.
- [ ] Unauthorized mutating commands cannot change the Check Enforcer result.
- [ ] The Check Enforcer status remains compatible with the existing `azure-sdk-tools` branch protection/ruleset.
- [ ] The implementation is enabled and validated on a real `azure-sdk-tools` pull request.
- [ ] After validation, `.github/workflows/event.yml` no longer invokes `azure/azure-sdk-actions@main` for Check Enforcer.
- [ ] Deployment/configuration and troubleshooting steps are documented.

## References

- Current workflow: https://github.com/Azure/azure-sdk-tools/blob/main/.github/workflows/event.yml
- Existing Check Enforcer behavior and test guidance: https://github.com/Azure/azure-sdk-actions/blob/main/docs/check-enforcer.md
- Retirement tracking: #15160

## Clean up

- Remove github/workflows/event.yml
- Archive 'azure/azure-sdk-actions' repo

Contributor guide

Open the contributing guide

Research direction

Start with the current .github/workflows/event.yml and docs/check-enforcer.md in azure-sdk-actions, then review the existing Check Enforcer unit tests and webhook payloads. Compare the app behavior against representative payloads, authorized and unauthorized commands, and Azure Pipelines filtering. Done means the required checks and commands work for azure-sdk-tools, tests and logging are present, configuration is documented, and the old workflow invocation is removed.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, github, github-actions
Domain
ci-cd, devops, security, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.