microsoft / microsoft/vscode-cmake-tools

CTest integration not picking up test names on first compile

Open
#3,358 18 comments 1 reaction 0 assignees View on GitHub
bug Feature: ctest Feature: Test Explorer
Dominant language
TypeScript
Stars
1.7k
Forks
546
Avg merge
2d 16h
Merged PRs (30d)
32

Description

### Brief Issue Summary

I have a project where I am managing tests through Catch2. The CTest integration can successfully find, run, and display the results from my tests, but I'm facing an odd issue where it only works on the second compile of the project.

The first compile and run of Ctest finds no tests because instead of running the tests with a command of the form:

`ctest -jxx -C Debug -T test --output-on-failure -R ^my_test_name-buildid`

it instead runs

`ctest -jxx -C Debug -T test --output-on-failure -R ^my_test_name_NOT_BUILT-buildid`

The build itself actually looks fine, running `ctest` with no arguments in the build directory will run all the tests.

What's odd is that if I recompile (say by selecting a different build variant) and recompile/retest, changing nothing else, it will correctly pick up and run all the tests.

I can't figure out where the extension is picking up the test names, for example grepping for NOT_BUILT in the build directory returns nothing.

### CMake Tools Diagnostics

_No response_

### Debug Log

_No response_

### Additional Information

_No response_

Contributor guide

Open the contributing guide

Research direction

Reproduce the first-compile CTest run in the build directory and compare its generated test filter with the second compile. Trace the CMake Tools CTest integration that produces the `ctest -R` command; done means the initial compile uses the test name rather than the `_NOT_BUILT` suffix and discovers the tests consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, typescript
Domain
developer-experience, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.