Nimblesite / Nimblesite/SharpLsp

Test Explorer: every run reports "No result reported" — discovered ids carry xUnit's unique-ID suffix

Open
#232 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug critical
Dominant language
TypeScript
Stars
132
Forks
5
Avg merge
6h 24m
Merged PRs (30d)
27

Description

Summary

Running any test from the Test Explorer against FluentValidation fails every test with

No result reported for FluentValidation.Tests.AbstractValidatorTester.When_the_validators_fail_then_the_errors_Should_be_accessible_via_the_errors_property (filter matched no test)

0/35 — nothing executed. The tree also renders every test with a 40-hex suffix in its label:

Can_replace_default_errorcode_resolver (15cf31e476658caedccc20a4b3fef02631fa4692)
CanValidateInstancesOfType_returns_false_when_comparing_against_some_other_type (2258dba774d9e515c4b…)
Default_error_code_should_be_class_name (fefa0728fbd9239435494fa7c4c1bbb801282c96)

Those two symptoms are the same defect. Run, Debug and Run-with-Coverage are all unusable on any project whose VSTest adapter emits this shape.

Root cause (confirmed on disk)

dotnet vstest … --ListFullyQualifiedTests does not always write a bare TestCase.FullyQualifiedName. With xunit.runner.visualstudio 2.2.0 (the version FluentValidation.Tests pins) it appends the test case's unique ID:

$ dotnet vstest bin/Debug/net8.0/FluentValidation.Tests.dll \
    --ListFullyQualifiedTests --ListTestsTargetPath:fqn.txt

$ head -3 fqn.txt
FluentValidation.Tests.AbstractValidatorTester.When_the_Validators_pass_then_the_validatorRunner_should_return_true (2bd9e5a56e8ea7c7114cb93b1c19bd50a55b7bf4)
FluentValidation.Tests.AbstractValidatorTester.When_the_validators_fail_then_validatorrunner_should_return_false (3695ae1728b33597f39bc187619010fc9c8b9fd1)
FluentValidation.Tests.AbstractValidatorTester.When_the_validators_fail_then_the_errors_Should_be_accessible_via_the_errors_property (4159b661adc378e56bd9b129de348febd7bf97e3)

813 of 816 listed names carry the suffix. parseFullyQualifiedTestList takes every line verbatim (src/editors/vscode/src/test-discovery.ts:135-138) — deliberately, because F# backtick names contain spaces and no shape filter is admissible — so the suffix becomes part of the test id.

From there the whole run path is poisoned:

Stage Value Result
makeTestItem (testing.ts:347) label = last dotted segment Method (4159b661…) in the tree
filterClause (test-filter.ts:32) FullyQualifiedName=…Method \(4159b661…\) parens escaped per [TEST-FILTER-ESCAPE]; no test has that name
parseTrxReport (test-trx.ts:146-153) keys on TestMethod/@className + . + @name bare FQN, never the suffixed one
reportMissing (test-reporting.ts:82) id absent from the TRX map No result reported for … (filter matched no test)

So the filter can never match, and even if it did the TRX key would not line up with the id.

The repo's own fixtures use xunit.runner.visualstudio 2.8.2, which emits bare FQNs — which is why the whole [TEST-DISCOVERY-FQN] e2e suite is green while a real-world project is completely broken.

Reproduction

  1. Open src/fixtures/real-world/fluentvalidation (or any project on xunit.runner.visualstudio 2.2.0).
  2. Open the Testing view and refresh.
  3. Observe the (40-hex) suffix on every test label.
  4. Press ▶ on the root. Every test errors with No result reported for ….

Expected

  • Test ids are the bare Namespace.Class.Method FQN, whatever the adapter appended.
  • Labels show the method name alone.
  • --filter matches and TRX outcomes attribute per test.

Notes for the fix

The suffix must be distinguished from names that legitimately end in parentheses — [TEST-DISCOVERY-FQN] requires Cs.Nunit.Fixtures.CalculatorTests.Adds_Case(2,2,4) to round-trip unchanged. The two differ: the adapter suffix is preceded by a space and its content is hex only.

Collapsing several xUnit theory rows onto one bare FQN is correct and already the documented behaviour ("xUnit [Theory] … no row data"); parseFullyQualifiedTestList already de-duplicates.

Environment

  • Windows 11, .NET SDK 10.0.303
  • SharpLsp branch fixloading
  • FluentValidation.Tests: <TargetFrameworks>net8.0;net9.0</TargetFrameworks>, xunit 2.2.0, xunit.runner.visualstudio 2.2.0, Microsoft.NET.Test.Sdk 16.0.1

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/editors/vscode/src/test-discovery.ts at parseFullyQualifiedTestList, then trace the ids through testing.ts, test-filter.ts, test-trx.ts, and test-reporting.ts. Run the TEST-DISCOVERY-FQN e2e suite and reproduce with src/fixtures/real-world/fluentvalidation. Done means adapter-suffixed names become bare FQNs without changing legitimate parenthesized names, filters match, and TRX outcomes report correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
developer-experience, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.