microsoft / microsoft/nodejstools

Mocha "before all" hooks run before each test when running from Test Explorer

Open
#1,604 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug test-adapter
Dominant language
C#
Stars
1.8k
Forks
355
PR merge metrics
No merged PRs in 30d

Description

Expected Behavior

When running Mocha tests from the VS2017 Test Explorer window, any "before all" hooks should be run once for the entire session (and any state created by the hooks should persist across all tests). Likewise, any "after all" hooks should only run once all tests are complete.

Actual Behavior

Firstly, global hooks are not discovered automatically unless the file describing the hooks is required by all test files. Secondly, any global "before all" and "after all" hooks end up running once per test as opposed to once per session. In my project, I am using a global "before all" hook to call Gulp and compile my TypeScript files before testing. When running from the command line, Gulp runs once before the testing session and then all the tests run. When running from NTVS' Test Explorer adapter, however, Gulp runs before each described test, causing the duration of each test to be upwards of 20 seconds as opposed to the milliseconds they take on the command line.

  • NTVS Version: 1.3.50417.1
  • Visual Studio Version: VS2017 Version 15.2 (26430.6)
  • Node.js Version: 7.7.1
Steps to Reproduce
  1. Set up a test file which describes at least two sample tests
  2. Set up a global hooks file alongside the test file which defines a global before hook
  3. Run mocha from the command line to observe that the hook runs once followed by both tests (proper behavior)
  4. Run all tests from the Visual Studio Test Explorer to observe first that the two tests run without the hook being run at all
  5. Have the test file require the hook file
  6. Run all tests again from the Visual Studio Test Explorer and observe that the hook is now run twice, once before each test

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Mocha execution and discovery path used by the Visual Studio Test Explorer adapter, comparing it with the command-line session behavior described in the issue. Reproduce the case with a test file containing two tests and a separate global hooks file. Done means global hooks are discovered automatically and each before/after-all hook runs once per test session.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
developer-experience, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.