Azure / Azure/azure-dev

Review and optimize Copilot PR reviewer configuration for azure-dev

Open
#7,318 1 comment 1 reaction 1 assignee Claimed by @rajeshkamal5050 View on GitHub
area/code-improvements engineering item
Dominant language
Go
Stars
569
Forks
364
Avg merge
2d 19h
Merged PRs (30d)
136

Description

## Goal

Audit and configure the Copilot PR reviewer to get the most value from automated reviews on this repo. PR #7223 showed the reviewer generating false positives due to missing repo context — #7315 / #7316 address one specific case (modern Go patterns), but we should do a broader review.

## Areas to investigate

### 1. Instruction files coverage
- **Current state**: Only \.github/instructions/extensions.instructions.md\ exists (subscription tenant guidance), plus #7316 (pending, Go patterns)
- **Gaps to evaluate**:
- Do we need a top-level \.github/copilot-instructions.md\ with repo-wide review guidance (architecture, IoC patterns, error handling conventions)?
- Should key patterns from \cli/azd/AGENTS.md\ (action-based commands, IoC container, error handling with \ErrorWithSuggestion\) be surfaced in instruction files?
- Are there other language/path-scoped instructions needed (e.g., Bicep templates, TypeScript extensions)?

### 2. Review behavior tuning
- **False positive patterns**: Catalog recurring false positives beyond #7223 — are there other common categories we should preempt?
- **Comment density**: Is the reviewer too noisy or too quiet? Should we adjust the review scope (e.g., security-only, bugs-only)?
- **Duplicate comments**: PR #7223 had 4 identical comments for the same issue on different lines — investigate if there's a way to consolidate

### 3. Repo settings
- Review the repo-level Copilot code review settings (Settings → Copilot → Code review)
- Confirm the reviewer is using the right base branch for instruction file loading
- Check if org-level policies affect behavior

### 4. Instruction file size budget
- The reviewer reads only the first ~4K characters per instruction file (per [GitHub docs](https://docs.github.com/en/copilot/tutorials/use-custom-instructions))
- Plan the instruction file strategy to stay within limits while covering the most impactful patterns

## Success criteria
- Reduced false positive rate on Go PRs
- Key repo conventions (IoC, action pattern, error handling) reflected in reviewer context
- Documented instruction file strategy so the team knows where to add future guidance

## References
- PR with false positives: #7223
- Existing fix for Go patterns: #7315 / #7316
- AGENTS.md: \cli/azd/AGENTS.md\
- GitHub docs on custom instructions: https://docs.github.com/en/copilot/tutorials/use-custom-instructions

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.