boostorg / boostorg/test

Missing formats for --list_content

Open
#397 0 comments 2 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
213
Forks
149
Avg merge
18h 17m
Merged PRs (30d)
2

Description

There are currently 2 formats for `--list_content`:

1. The human-readable format, great visually finding a test
2. The dot format, great for visualizing the test-suite

Over the years, I found 2 formats to be missing:

### List of test-cases

For simple tools or integration with external test runners like CTest, it would be super useful to have an output listing all testcases in an easy to consume way:

```
testsuite1/testcase1
testsuite1/testcase2
testsuite2/testcase3
```

Generating this information from the existing formats is rather tedious.

Use cases:
* Register testcases as separate tests external test runners, such as CTest
* Writing custom tools that work only on test cases, such as bisection of test cases, running them with GNU parallel, or running them individually in various docker containers.
* Use fuzzy search to quickly find a testcase and run it `./test --list_content=LIST | fzf | xargs ./test -t`

### Detailed machine-readable output

For advanced tools, a detailed dump of the test tree would be useful.
JSON would be suitable as it can easily be generated without special libraries.

```json
{
"type": "suite",
"name": "testsuite1",
"fileName": "...",
"fileLine": 21,
"labels": [],
"children": [
{
"type": "case",
"name": "testcase1",
"fileName": "...",
"fileLine": 23,
"labels": []
},
{
"type": "case",
"name": "testcase2",
"fileName": "...",
"fileLine": 33,
"labels": []
}
]
}
```

The DOT format includes some of this information in a less structured way as most of it is encoded in the label.

Use cases:
* Custom linters
* Merging the detailed test setup with external data
* Intermediate format to generate other custom formats

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.