microsoft / microsoft/GitHub-Copilot-for-Azure

Add lightweight routing-only test to validate skill selection across models on every PR

Open
#2,166 2 comments 0 reactions 0 assignees View on GitHub
enhancement integration-test untriaged
Dominant language
Python
Stars
250
Forks
204
Avg merge
1d 12h
Merged PRs (30d)
67

Description

### Problem

Skill selection is only fully tested through slow, comprehensive test suites, limiting our ability to catch regressions early on every PR.

### Desired Solution

Add a light, fast test that validates routing-only/skill selection for all models, omitting full content quality checks. This can run on every PR without slowing down the workflow.

### Alternatives and Workarounds

Continue only using full test suites, or try to speed up those suites but risk missing skill selection bugs on PRs.

### Impact

Faster detection of routing/skill selection errors; prevent regressions from being merged due to lack of early feedback; allows full test suites to focus on deep correctness.

### Additional Context

Request originated in [Issue #1647](https://github.com/microsoft/GitHub-Copilot-for-Azure/issues/1647). Intended to complement, not replace, full suite coverage.

Contributor guide

Open the contributing guide

Research direction

Locate the existing full skill-selection test suites and the pull-request workflow that runs them. Start by identifying how models are enumerated and how routing-only behavior can be isolated; done means a fast test covers all models on every PR without running full content-quality checks.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.