microsoft / microsoft/nodejstools

Running tests with vstest.console.exe on a .net core unit test project fails.

Open
#2,342 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C#
Stars
1.8k
Forks
355
PR merge metrics
No merged PRs in 30d

Description

While executing unit tests in a generated unit test project from the command line an exception occurs.
In my case, the unit tests are all C# tests, but the referenced Asp.Net core project contains TypeScript Mocha tests.

Note, I was able to override the TestFrameworks path using a RunSettings file, however it would be desirable for this not to be required.

Expected Behavior

Unit tests to be executed without an exception and without requiring a runsettings file.

Actual Behavior

An example of the output:

Microsoft (R) Test Execution Command Line Tool Version 16.11.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
An exception occurred while invoking executor 'executor://nodejstestexecutor/v1': Unable to find test framework folder. Tried: "C:\dev\AspNetCoreTest\Active\Web\UnitTests\AspNetCoreTest.Tests\bin\Debug\netcoreapp3.1\..\_common\TestFrameworks"
Stack trace:
   at Microsoft.NodejsTools.TestFrameworks.TestFrameworkDirectories.GetFrameworkDirectories(String testFrameworkRoot)
   at Microsoft.NodejsTools.TestAdapter.TestFrameworks.FrameworkDiscoverer..ctor(String testFrameworkRootDirectory)
   at Microsoft.NodejsTools.TestAdapter.TestExecutorWorker..ctor(IRunContext runContext, IFrameworkHandle frameworkHandle)
   at Microsoft.NodejsTools.TestAdapter.NetCoreTestExecutor.RunTests(IEnumerable`1 sources, IRunContext runContext, IFrameworkHandle frameworkHandle)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.RunTestsWithSources.InvokeExecutor(LazyExtension`2 executor, Tuple`2 executorUriExtensionTuple, RunContext runContext, IFrameworkHandle frameworkHandle)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.RunTestInternalWithExecutors(IEnumerable`1 executorUriExtensionMap, Int64 totalTests)

  Passed QueryForJsonStringAsyncTest [810 ms]
  Passed GetWhereClauseTest [68 ms]
  Passed GetCriteriaTest [< 1 ms]
  Passed SimplifyCriteriaTest [1 ms]
Results File: C:\dev\AspNetCoreTest\Active\Web\UnitTests\AspNetCoreTest.Tests\bin\Debug\netcoreapp3.1\TestResults\ME_MYBOX_2021-08-16_17_50_50.trx

Test Run Failed.
Total tests: 4
     Passed: 4
 Total time: 1.5941 Seconds

The problem seems to be from the TestFrameworkDirectories class's GetTestframeworkFolderRoot method using the unit test dll's location and then appending ..\_common to the path.

Here is the command line I executed:

"%ProgramFiles(x86)%\Microsoft Visual Studio\2019\Professional\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" AspNetCoreTest.Tests.dll /Platform:X64 /Logger:Trx /InIsolation /Logger:Console"

Here is the RunSettings file that I used to force the path:

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
	<JavaScriptUnitTest>
		<TestFrameworkRoot>C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\Extensions\Microsoft\NodeJsTools\TestAdapter\TestFrameworks</TestFrameworkRoot>
	</JavaScriptUnitTest>
</RunSettings>
Steps to Reproduce
  1. Create an Asp.Net core project
  2. Setup Mocha unit testing following the instructions located here: https://docs.microsoft.com/en-us/visualstudio/javascript/unit-testing-javascript-with-visual-studio?view=vs-2019&tabs=mocha
  3. Add a library class to the project with a simple method
  4. Generate a unit test for the library class in a new test project
  5. Execute the unit test via the command line

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with the provided vstest.console.exe command and the generated .NET Core test project. Start with TestFrameworkDirectories.GetTestframeworkFolderRoot and GetFrameworkDirectories, then verify the fix by running the same command without a RunSettings file. Done means the C# tests execute without the nodejstestexecutor path exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, node.js, typescript
Domain
testing-qa, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.