llvm / llvm/llvm-project

Many skipped lldb API tests show only "Unsupported configuration" in lit XML results

Open
#214,714 4 comments 1 reaction 0 assignees View on GitHub
lldb test-suite
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

In GitHub CI we have lit produce results in a JUnit XML file and we make nice reports out of it. Sometimes I want to know if a specific test ran or not, so I download these files from the build artifacts.

For example from https://github.com/llvm/llvm-project/actions/runs/31171738943?pr=214515. Note that you need to go to the summary page to see the downloads, they're at the bottom of the page.

The XML can include a reason for being skipped, but many lldb tests are just this generic message:
```
$ cat test-results.bi7v42b2.xml | grep "
25
16
9
9
5
5
4
3
3
3
2
2
2
2
2
2
1
1
1
1
1
1
1
1
1
1
```
Somehow the others make it into the XML so I think we can figure out the rest of them.

It's really useful to know which part of the configuration caused the test to skip. It helps to confirm the CI machines have the right environment.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the generic "Unsupported configuration" entries in lit's JUnit XML output for the LLDB API tests, using the linked GitHub Actions results as a reference. Trace how skip reasons are generated and ensure the XML reports the configuration feature that caused each skip while preserving the existing specific messages.

Written by the indexing model from the issue text.

Assessment

Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.