microsoft / microsoft/GitHub-Copilot-for-Azure
Copilot PR reviewer creates infinite fix→review→fix cycle on skill PRs
- Dominant language
- Python
- Stars
- 250
- Forks
- 204
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 67
Description
## Problem
Every push to a skill PR triggers `copilot-pull-request-reviewer`, which finds new nitpicks. We fix them, push, and the reviewer fires again with more comments. This creates an unbreakable whack-a-mole cycle that blocks PRs from getting clean.
**Example from PR #1636:**
- Started with 24 review threads (copilot-reviewer + JonG)
- Fixed all, pushed → 6 new copilot comments appeared
- Fixed those, pushed → 7 MORE new copilot comments appeared
- Comments are increasingly stylistic/pedantic (e.g. "variable name not defined in snippet" on reference docs that aren't meant to be compilable)
Same pattern on #1637 and #1642.
## Root Cause
The copilot-reviewer treats **reference documentation** (markdown with embedded Bicep snippets) the same as **production code**. It flags things like:
- Undefined variables in illustrative snippets (`uamiClientId` referenced but not declared as param)
- Design choices as bugs (`minReplicas: 0` vs `1`)
- Wording precision ("copy" vs "create" for recipe modules)
- PR description not matching issue deliverables exactly
These are valid observations for **code review** but inappropriate strictness for **reference documentation templates** that are intentionally simplified.
## Impact
- PRs #1636, #1637, #1642 have been through 3+ fix cycles each
- Each cycle takes ~15-30 min (read comments, fix, push, wait for CI, resolve threads)
- Human reviewers (JonG, Simon) already approved the substance — copilot keeps finding new surface-level issues
## Proposed Solutions
1. **Configure copilot-reviewer to be less strict on `references/` and `templates/` paths** — these are docs, not code. Lower the sensitivity or skip certain categories of findings.
2. **Add a `copilot-review: skip` label or comment** that suppresses re-review on follow-up pushes after human approval
3. **Scope the reviewer to only review the initial diff**, not re-review the entire file on every push
4. **Separate code-quality checks from doc-quality checks** — different rules for `.md` files vs `.ts`/`.bicep` files
## Current Workaround
We're triaging comments into "fix" vs "reply-and-resolve as intentional", then batch-pushing + immediately resolving all threads via GraphQL API before the next review cycle can pile up. This works but is manual and fragile.
cc @paulyuk @simonjj
Contributor guide
Assessment
This issue has not been assessed yet.