catchorg / catchorg/Catch2

catch_discover_test PROPERTIES does not work with lists

Open
#2,424 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
21.5k
Forks
3.5k
Avg merge
3d 16h
Merged PRs (30d)
2

Description

**Describe the bug**

When still using `ParseAndAddCatchTests`, I had code like this to set multiple environment variables for running the tests:

```cmake
list(APPEND _environment_vars foo=bar)
list(APPEND _environment_vars f00=b4r)
ParseAndAddCatchTests(${test})
get_target_property(sub_tests ${test} ParseAndAddCatchTests_TESTS)
set_tests_properties(${sub_tests} PROPERTIES ENVIRONMENT "${_environment_vars}")
```

When using `catch_discover_tests`, I tried to replace the last three lines with:

```cmake
catch_discover_tests(${test} PROPERTIES ENVIRONMENT "${_environment_vars_list}")
```

But after observing some indirect errors a while later, I noticed that only the very first environment variable was actually exported.

Looking under the hood, the problem seems to be that `PROPERTIES` are stored as a list itself, so that lists as value / semicolons in values will lead to problems.

My workaround currently is to add `ENVIRONMENT;` for each environment variable, resulting in:

```cmake
string(REPLACE ";" ";ENVIRONMENT;" _environment_vars_list "${_environment_vars}")
catch_discover_tests(${test} PROPERTIES ENVIRONMENT "${_environment_vars_list}")
```

**Expected behavior**

`catch_discover_tests(${test} PROPERTIES ENVIRONMENT "${_environment_vars_list}")` should work even if there are multiple environment variables specified. This interface would be consistent with `set_tests_properties`.

Alternatively, this differing behavior for `PROPERTIES` should be documented, e.g., inside `docs/cmake-integration.md`, if possible with a suggested workaround.

**Platform information:**

- Catch version: **v2.13.9**

Contributor guide

Open the contributing guide

Research direction

Start with the catch_discover_tests PROPERTIES handling and compare it with the set_tests_properties behavior described in the issue. Reproduce the multiple-environment-variable case, then inspect docs/cmake-integration.md for the relevant interface and determine whether the behavior should be fixed or documented with the workaround. Done means multiple ENVIRONMENT entries are preserved or the documented behavior and workaround are clear.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp
Domain
build-system, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.