kernelci / kernelci/kernelci-pipeline
Report test configuration details in the nodes
- Dominant language
- Python
- Stars
- 11
- Forks
- 40
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 14
Description
This issue is to initiate an open discussion regarding the possibility of providing users with more comprehensive information about the specific configurations of test suites through the KernelCI nodes.
Tests may be executed with different parameters across different platforms (depending on factors such as hardware configuration). Thus, it's important to provide details on the configuration/parameters used on a specific test run. This enables users to know how the test was conducted on each platform and to easily identify if a test failure correlates with the specific configuration in use.
This may become particularly relevant with the addition of support for running kselftests with parameters (see https://github.com/Linaro/test-definitions/pull/511).
Some possible solutions:
- Consider adding extra node fields to report the command used for the test and/or the parameters applied (where applicable).
- Ensure that test suites running with different parameters are assigned distinct names or IDs. This may be challenging to enforce as a rule, and users may still need to refer to the YAML files for detailed test information.
Some test suites can be harder to describe than others, so I feel like the challenge is to find a way to effectively describe the test configuration that works for different test suites (maybe a link to the test job template used is enough?).
Any thoughts on this?
Contributor guide
No contributing guide indexed for this repository
Research direction
Review the YAML files and the kselftests parameter work in PR #511, then examine how test suites are represented in KernelCI nodes. The work is done when users can distinguish runs with different parameters and inspect the command, parameters, or test job template used for each run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- yaml
- Domain
- testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100