dotnet / dotnet/aspnetcore

dotnet watch test should only run changed tests

Open
#25,194 6 comments 19 reactions 0 assignees View on GitHub
affected-few area-commandlinetools enhancement feature-hot-reload severity-minor
Dominant language
C#
Stars
38.4k
Forks
10.9k
Avg merge
2d 6h
Merged PRs (30d)
290

Description

The issue #10366 was marked as stale prematurely. I think it still makes sense to implement.

Right now, the `dotnet watch test` runs all tests every time even if only one file has changed.

The test runner can already provide code coverage. It could use this coverage information to determine if some code that has been modified, should trigger re-running a specific test.

Most other test runners already do this, and it is super useful. For instance, Live Unit Testing in Visual Studio Professional, Jest does it within JavaScript, Ava seems to do it, and ReSharper's DotCover does it as well.

Right now, I have to run my whole test suite every time I change a few things. That's a huge time consumer! It forces me to use Visual Studio instead of Visual Studio Code for now, and I'd love to make the switch.

*I think it is important that coverage files are still supported. That way, Visual Studio Code extensions like Coverage Gutters still works well in watch mode.*

# Flow
To explain this with an example, I'll describe it as a flow of steps.

## 1. I make a class and a test
```csharp
public class SomeClass {
public bool SomeCondition {get;set;}

public A() {
if(SomeCondition) {
Console.WriteLine("A1");
} else {
Console.WriteLine("A2");
}
}
}
```

```csharp
[TestClass]
public class SomeClassTest {
[TestMethod]
public void TestA1() {
var obj = new SomeClass() { SomeCondition = true };
obj.A();
}

[TestMethod]
public void TestA2() {
var obj = new SomeClass() { SomeCondition = false };
obj.A();
}
}
```

## 2. I run the tests in watch mode
The test runner now internally keeps track of the coverage. In other words, half of method `A` is now covered by `TestA1`, and half by `TestA2`.

## 3. I modify some of the code
I modify the original class to be the following (changing the `Console.WriteLine("A1")` to `Console.WriteLine("foo")`:
```csharp
public class SomeClass {
public bool SomeCondition {get;set;}

public A() {
if(SomeCondition) {
Console.WriteLine("foo"); //this line was changed
} else {
Console.WriteLine("A2");
}
}
}
```

## 4. The test runner runs affected tests
Now, the test runner detects that only covered paths by the test `TestA1` has been changed, so it only runs the `TestA1` test.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.