Determine metrics for tracking CCR quality

How do we know if anything we're doing actually helps?

Core questions:
1. Are humans doing less work thanks to CCR?
2. How much of Copilot comments are actually useful?
3. Is CCR helping us catch critical issues?

Metrics we could track to answer each question:

## Are humans doing less work?

- \# human comments per PR over time (normalized based on PR type: bug fix, refactor, new feature, doc update, etc.
- Consideration: we should track if Copilot had a review round before counting human comments as 'Copilot misses'. E.g., Copilot may review a PR when its opened, but it wouldn't re-review later commits unless re-requested. Human comments on rounds without any Copilot review maybe shouldn't be taken into consideration here?
- overall time to complete a PR
- number of commit/iterations on a PR?

## Are Copilot comments useful?

- \# Copilot comments that are "acted on" - a human responds positively, 'resolved' rate (for example, AI analyzes at merge time which comments were addressed by the author). Normalize based on comment type (bug, nitpick, critical)
- \# comments that are ignored or rejected
- sentiment response to comments (waiting for better data)

## Are we catching critical issues?

- \# bug-fix PRs merged over time as proxy for \# bugs getting merged in
- ^ \# critical Copilot comments that get acted on

## key

- filtering/normalizing across diff PR types and comment types
- filtering low-signal noise - nitpicks, "lgtm"/short comments, comments from non-contributors
- avoid overfitting - prompt edits should be generalized to entire repo, and not redundant to something that already exists