IvanMurzak / IvanMurzak/Unity-MCP
tests-run: TotalTests ignores all filters (counts entire test tree)
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 4.3k
- Forks
- 379
- Avg merge
- 6h 52m
- Merged PRs (30d)
- 17
Description
tests-run reports TotalTests as the size of the entire test tree for the given testMode, ignoring testAssembly / testNamespace / testClass / testMethod. PassedTests / FailedTests / SkippedTests count only executed tests, so the summary is internally inconsistent whenever a filter is set.
Repro
Unity 6000.3.5f1, com.ivanmurzak.unity.mcp 0.87.0. Project has 941 tests in EditModeTests.dll.
tests-run { testMode: EditMode, testAssembly: "EditModeTests", testClass: "SongPipelineCacheTests" }
→ { Status: "Passed", TotalTests: 942, PassedTests: 1, FailedTests: 0, SkippedTests: 0 }
One test ran, but TotalTests reports the whole EditMode tree (941 project tests + 1 stub from com.unity.addressables). The same 942 appears for an unfiltered run, a single-assembly run, and this single-class run — the value is constant regardless of filter.
Cause
TestResultCollector.RunFinished — Editor/Scripts/API/Tool/Tests/TestResultCollector.cs:105:
_summary.TotalTests = CountTests(result.Test);
result.Test roots the full test tree, and CountTests walks every leaf with no filter awareness.
RunStarted (line 86) already sets a correct filtered count from testsToRun:
_summary.TotalTests = testCount; // correct, filtered
Line 105 then overwrites it with the unfiltered count.
Impact
TotalTests != PassedTests + FailedTests + SkippedTestswhenever a filter is set, which reads as missing or silently-skipped tests.- Callers can't use
TotalTeststo confirm the intended set actually ran. - Particularly confusing for agent/LLM callers, which reasonably treat the summary as self-consistent and report a false discrepancy to the user.
Suggested fix
Drop the reassignment at line 105 and keep the RunStarted value, or count against the filtered set — Tool_Tests.CountFilteredTests (Tests.Run.cs) already implements filter-aware counting and is used for pre-run validation.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in Editor/Scripts/API/Tool/Tests/TestResultCollector.cs, especially RunStarted and RunFinished, and compare the TotalTests assignments. Review Tool_Tests.CountFilteredTests in Tests.Run.cs, then run the filtered tests-run cases from the issue. Done means TotalTests reflects the selected tests and remains consistent with the executed-test counts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, unity
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 86/100