microsoft / microsoft/mcp

Improve CI testing

Open
#2,755 1 comment 0 reactions 0 assignees View on GitHub
Central-EngSys enhancement server-Azure.Mcp tools-Core
Dominant language
C#
Stars
3.7k
Forks
624
Avg merge
2d 20h
Merged PRs (30d)
220

Description

There are a few things which currently slow down our CI times, which cause long CI times and potential cases where CI times out as certain steps (such as testing) take too long. We can improve this with a few changes.

- `Run unit tests` step runs all test projects always. This is a bit excessive as changing something in the Storage tools shouldn't require the entire repos unit tests to run. This step should be scoped to the projects changed (unless the projects changed are Core libraries or `/eng`, in which case the current behavior is correct).
- `Run unit tests` and `Run recorded tests` should be merged into a single `Run tests` step. The two steps were kept separate when `UnitTests` and `LiveTests` were different projects. These have been merged and other infrastructure has been improved so that these no longer need to be separate things. (The first point above should be completed first before this is done)
- Each live test class results in a new Test Proxy being started. This may need to be kept but it also means that every live test class starts an application that isn't cheap to maintain. This also results in more resource contention in the runners.
- Improve test run outputs to be more actionable. Right now, most output is just indicating that tests passed or is a few large JSON blobs. This makes troubleshooting which tests failed and why much more difficult. Test logging output should be cleaned up.

Contributor guide

Open the contributing guide

Research direction

Start by locating the CI definition containing the `Run unit tests` and `Run recorded tests` steps, then review how changed projects and Core or `/eng` changes are detected. Check the existing test-project and Test Proxy setup before assessing the step merge, proxy reuse, and logging changes. Done means CI runs the appropriate tests, combines the test steps, avoids unnecessary proxy startup where possible, and produces actionable failure output.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
ci-cd, testing
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.