microsoft / microsoft/GitHub-Copilot-for-Azure
Compute a per skill "coverage" list for integration tests
- Dominant language
- Python
- Stars
- 250
- Forks
- 204
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 67
Description
Good test coverage indicates the tests cover all possible ways the test subject can be involved. For traditional software, it means every line of code that can be executed. For skills, it means our test prompts cover enough scenarios where the skill is expected to help the agent. We can aggregate the test prompts and show a per skill list of all the test prompts being used.
Display this per skill list of prompts in the dashboard allows us to review the covered scenarios for each skill and ask the following question: "Is there another prompt that should have resulted in using the skill to accomplish something that hasn't been covered by existing tests?" This is assuming each test case runs one prompt. This happens to be the case for all existing integration tests and there is no foreseeable plan to change that.
Contributor guide
Research direction
The issue names no files, tests, or entry points. Start by locating the existing integration-test prompt aggregation and dashboard skill views; verify that each test case contributes one prompt, then confirm the dashboard shows a per-skill prompt list.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- analytics, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100