IvanMurzak / IvanMurzak/Unity-MCP

tests-run: TotalTests ignores all filters (counts entire test tree)

Open Beginner friendly
#950 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement help wanted
Dominant language
C#
Stars
4.3k
Forks
379
Avg merge
6h 52m
Merged PRs (30d)
17

Description

tests-run reports TotalTests as the size of the entire test tree for the given testMode, ignoring testAssembly / testNamespace / testClass / testMethod. PassedTests / FailedTests / SkippedTests count only executed tests, so the summary is internally inconsistent whenever a filter is set.

Repro

Unity 6000.3.5f1, com.ivanmurzak.unity.mcp 0.87.0. Project has 941 tests in EditModeTests.dll.

tests-run { testMode: EditMode, testAssembly: "EditModeTests", testClass: "SongPipelineCacheTests" }
→ { Status: "Passed", TotalTests: 942, PassedTests: 1, FailedTests: 0, SkippedTests: 0 }

One test ran, but TotalTests reports the whole EditMode tree (941 project tests + 1 stub from com.unity.addressables). The same 942 appears for an unfiltered run, a single-assembly run, and this single-class run — the value is constant regardless of filter.

Cause

TestResultCollector.RunFinishedEditor/Scripts/API/Tool/Tests/TestResultCollector.cs:105:

_summary.TotalTests = CountTests(result.Test);

result.Test roots the full test tree, and CountTests walks every leaf with no filter awareness.

RunStarted (line 86) already sets a correct filtered count from testsToRun:

_summary.TotalTests = testCount;   // correct, filtered

Line 105 then overwrites it with the unfiltered count.

Impact

  • TotalTests != PassedTests + FailedTests + SkippedTests whenever a filter is set, which reads as missing or silently-skipped tests.
  • Callers can't use TotalTests to confirm the intended set actually ran.
  • Particularly confusing for agent/LLM callers, which reasonably treat the summary as self-consistent and report a false discrepancy to the user.

Suggested fix

Drop the reassignment at line 105 and keep the RunStarted value, or count against the filtered set — Tool_Tests.CountFilteredTests (Tests.Run.cs) already implements filter-aware counting and is used for pre-run validation.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in Editor/Scripts/API/Tool/Tests/TestResultCollector.cs, especially RunStarted and RunFinished, and compare the TotalTests assignments. Review Tool_Tests.CountFilteredTests in Tests.Run.cs, then run the filtered tests-run cases from the issue. Done means TotalTests reflects the selected tests and remains consistent with the executed-test counts.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, unity
Domain
testing-qa
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
86/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.